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Why  GAO  Did  This  Study 

DOD’s  colleges  and  universities  that 
provide  JPME,  including  their  research 
institutions,  are  intended  to  develop 
military  personnei  throughout  their 
careers  by  broadening  them 
inteliectuaily  and  fostering 
coliaboration  across  the  military 
services.  JPME  research  institutions 
generaliy  provide  studies  and  analysis 
research  that  can  support  academic 
programs  or  inform  DOD  policymakers. 

The  National  Defense  Authorization 
Act  for  FY  2013  mandated  GAO  to 
review  JPME  research  institutions. 
GAO’s  report  (1)  describes  how  JPME 
research  institutions  have  changed  in 
number,  funding,  and  size;  (2) 
evaluates  the  extent  to  which  DOD 
assesses  JPME  research  institution 
performance:  and  (3)  evaluates  the 
extent  to  which  DOD  coordinates  the 
research  requests  of  these  and  other 
DOD-funded  research  organizations. 
GAO  identified  and  examined  the 
20  JPME  research  institutions  that 
conduct  research  as  their  primary 
mission  and  have  dedicated  personnel. 
GAO  reviewed  DOD  documents  and 
interviewed  officiais  on  changes  at  the 
20  institutions  and  how  they  are 
overseen,  as  weli  as  the  processes  to 
coordinate  their  research  activities  and 
those  of  14  other  DOD-funded 
research  organizations  GAO 
determined  conduct  research  activities. 

What  GAO  Recommends 

GAO  recommends  that  DOD  take 
actions  to  define  the  role  of  JPME 
research  institutions,  assign 
responsibilities  for  assessing 
performance,  and  establish  a 
mechanism  to  coordinate  studies  and 
anaiysis  research  requests.  DOD 
concurred  with  the  recommendations. 

View  GAO-14-216.  For  more  information, 
contact  Johana  Ayers  at  (202)  512-5741  or 
ayersj@gao.gov. 


What  GAO  Found 

Joint  Professionai  Miiitary  Education  (JPME)  research  institutions,  particuiariy  at 
the  Nationai  Defense  University,  experienced  growth  in  number,  funding,  and 
size  in  terms  of  staffing  ieveis  from  fiscai  year  (FY)  2007  through  FY  201 1 ,  but 
the  number  of  institutions  as  weii  as  funding  and  staffing  ieveis  deciined  over  the 
past  2  years.  For  exampie,  totai  funding  for  JPME  research  institutions  increased 
from  $30.8  miiiion  in  FY  2007  to  $47.7  miiiion  in  FY  2011,  but  subsequentiy 
decreased  to  $40.6  miiiion  in  FY  2013.  GAO  identified  severai  factors  that 
contributed  to  these  institutions’  growth,  inciuding  increases  in  funding  provided 
by  outside  organizations  for  research  and  the  creation  of  new  research 
institutions.  Department  of  Defense  (DOD)  officiais  reported  that  DOD-wide 
budget  reductions,  inciuding  the  effects  of  sequestration,  contributed  to 
decreases  in  the  number,  size,  and  funding  for  JPME  research  institutions. 

The  extent  to  which  DOD  can  assess  the  performance  of  JPME  research 
institutions  is  iimited  by  the  iack  of  a  comprehensive  framework  to  systematicaiiy 
assess  their  performance  in  meeting  professionai  miiitary  education  and  other 
departmentai  goais  and  objectives.  JPME  coiieges  and  universities  have  not 
consistentiy  estabiished  measurabie  goais  or  objectives  iinked  with  performance 
metrics  for  their  associated  research  institutions.  Best  practices  state  that 
achieving  resuits  in  government  requires  a  framework  with  measurabie  goais  and 
objectives  and  metrics  to  assess  progress.  Further,  oversight  mechanisms  for  the 
colieges  and  universities,  such  as  accreditation  processes,  focus  on  the  quality  of 
JPME  academic  programs  and  not  on  the  research  institutions’  performance. 
There  is  no  DOD-wide  guidance  that  addresses  the  intended  role  of  the  research 
institutions  in  supporting  JPME  or  other  departmental  goals,  or  assigns 
responsibilities  for  conducting  reviews  of  them,  leaving  the  department  without  a 
basis  to  assess  the  institutions’  stated  mission  and  actual  performance  against 
planned  or  expected  results.  Therefore,  DOD  does  not  have  a  basis  to  assess 
the  institutions’  missions  and  performance  against  expected  results,  as  called  for 
by  best  practices.  Without  measurable  goals  and  objectives  linked  with 
performance  metrics,  and  clear  guidance  on  their  intended  roles  and  assignment 
of  oversight  responsibilities,  DOD  cannot  ensure  JPME  research  institutions  are 
effectively  accomplishing  their  missions. 

DOD  has  not  established  mechanisms  to  coordinate  requests  for  research 
conducted  by  JPME  research  institutions  and  other  DOD-funded  research 
organizations  because  there  is  no  requirement  to  do  so.  Although  many  of  these 
organizations  have  missions  to  conduct  research  in  similar  topic  areas,  DOD 
uses  a  variety  of  processes  to  request  studies  and  analysis  research. 

Specifically,  offices  within  the  Office  of  the  Secretary  of  Defense  and  the  military 
departments  each  have  their  own  separate  internal  processes  to  manage 
research  requests  and  do  not  participate  in  one  another’s  processes.  Best 
practices  on  managing  for  results  state  that  organizations  involved  in  similar 
missions  should  coordinate  and  share  information  to  avoid  unnecessary 
duplication  of  work.  At  a  time  of  constrained  budgets,  fragmentation  in  DOD’s 
approach  to  managing  its  research  requests  across  the  department  exposes 
DOD  to  the  risk  of  potential  overlap  of  studies  and  analysis  research. 
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U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


March  10,  2014 
Congressional  Committees 

As  operations  in  Iraq,  Afghanistan,  and  elsewhere  over  the  past  decade 
have  demonstrated,  military  personnel  must  think  critically,  communicate 
well,  conduct  themselves  with  integrity,  and  lead  others  in  difficult  and 
often  dangerous  situations  that  involve  forces  from  across  the  four  military 
services.  To  that  end,  the  Department  of  Defense  (DOD)  relies  on  joint 
professional  military  education  (JPME),  a  subset  of  professional  military 
education  (PME),''  to  educate  servicemembers  throughout  their  careers, 
broaden  their  knowledge,  improve  performance  during  joint  assignments, 
and  foster  collaboration  across  the  military  services.  In  1989,  a 
congressional  panel  undertaking  a  comprehensive  review  of  PME 
strongly  expressed  the  importance  of  JPME  in  guiding  intellectual  thought 
on  doctrine,  tactics,  strategy,  and  the  future  of  each  of  the  military 
services.^  DOD  has  colleges  and  universities  throughout  the  country  that 
not  only  provide  academic  instruction  in  PME  and  JPME,  but  also  conduct 
research.^  At  JPME-granting  colleges  and  universities,  research  is 
conducted  by  students  and  professors,  as  well  as  by  specifically 
dedicated  research  institutions.  These  JPME  research  institutions'^ 
primarily  conduct  studies  and  analysis  research  that  may  directly  support 
both  PME  and  JPME  academic  programs  by  providing  new  concepts  for 


^According  to  DOD  guidance,  officer  professional  military  education — both  service  and 
joint — provides  the  education  needed  to  complement  training,  experience,  and  self- 
improvement  to  produce  the  most  professionally  competent  (strategic-minded,  critical- 
thinking)  servicemembers.  Department  of  Defense,  Chairman  of  the  Joint  Chiefs  of  Staff, 
Officer  Professional  Military  Education  Policy  (OPMEP),  Chairman  of  the  Joint  Chiefs  of 
Staff  Instruction  (CJCSI)  1 800.01  D  (Washington,  D.C.:  July  15,  2009).  (Hereinafter  cited 
as  CJCSI  1800.01D.) 

^Committee  on  Armed  Services,  House  of  Representatives,  Report  of  the  Panel  on  Military 
Education  of  the  One  Hundredth  Congress,  101st  Cong.,  1st  sess..  No.  4,  (1989). 

^For  the  purposes  of  this  report,  we  refer  to  service  and  joint  colleges  and  universities  that 
are  accredited  by  the  Joint  Staff  to  provide  JPME  certification  as  JPME  colleges  and 
universities. 

'‘For  purposes  of  this  report,  we  refer  to  research  institutions  affiliated  with  service  and 
joint  colleges  and  universities  as  JPME  research  institutions. 
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the  classroom  or  DOD’s  broader  needs,  such  as  policy  development  and 
planning  efforts.® 

Given  its  importance  in  educating  military  leaders,  a  2010  report  from  the 
House  Armed  Services  Subcommittee  on  Oversight  and  Investigations 
concluded  that  PME,  of  which  JPME  is  a  subset,  warrants  more-frequent 
congressional  oversight.®  However,  our  prior  work^  has  found  that  past 
studies  of  PME  were  unable  to  fully  identify  the  costs  associated  with 
DOD’s  JPME  programs.  Specifically,  we  found  that  without  reliable  cost 
information,  decision  makers  could  be  hindered  in  determining  the  most- 
efficient  allocation  of  departmental  resources  for  JPME.  Moreover,  the 
Joint  Staff  has  expressed  concerns  about  considerable  budgetary  growth 
at  the  National  Defense  University  (NDU),  which  DOD  has  characterized 
as  the  premier  center  for  JPME.  For  example,  in  a  201 2  review,  the  Joint 
Staff  concluded  that  it  could  not  identify  a  clear  link  between  some  of  the 
activities  of  NDU’s  research  institutions  and  its  JPME  mission. 

The  National  Defense  Authorization  Act  for  Fiscal  Year  2013  mandated 
that  we  review  the  work  performed  by  JPME  research  institutions  in 
support  of  PME  and  DOD’s  broader  mission.®  In  this  report,  we  (1) 
describe  how  JPME  research  institutions  have  changed  in  number, 
funding,  and  size  and  the  factors  that  contributed  to  any  changes;  (2) 
evaluate  the  extent  to  which  DOD  is  assessing  the  performance  of  JPME 
research  institutions  in  meeting  PME  and  other  departmental  goals  and 
objectives;  and  (3)  evaluate  the  extent  to  which  DOD  coordinates 
research  requests  for  JPME  research  institutions  and  other  DOD-funded 
research  organizations. 

To  address  these  objectives,  we  included  in  the  scope  of  our  review  the 
intermediate-  and  senior-level  colleges  and  universities  that  provide 


^DOD  broadly  defines  studies  and  analysis  research  as  research  done  to  support  DOD 
policy  development,  decision  making,  alternative  approaches,  and  new  ideas  for  the  DOD 
community. 

^Subcommittee  on  Oversight  and  Investigations,  Committee  on  Armed  Services,  House  of 
Representatives,  Another  Crossroads?  Professional  Military  Education  Two  Decades  After 
the  Goldwater-Nichols  Act  and  the  Skelton  Panel,  H.R.  111-4  (2010). 

^GAO,  Joint  Military  Education:  Actions  Needed  to  Implement  DOD  Recommendations  for 
Enhancing  Leadership  Development,  GAO-14-29  (Washington,  D.C.:  Oct.  23,  2013). 

"Pub.  L.  No.  112-239,  §  547(b)  (2013). 
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JPME  certification — that  is,  NDU,  the  Army  War  College,  Army  Command 
and  General  Staff  College,  Air  University,  Naval  War  College,  and  Marine 
Corps  University.  Research  conducted  at  these  JPME  colleges  and 
universities  can  be  accomplished  by  research  institutions  associated  with 
the  colleges  and  universities  as  well  as  by  students  and  professors  as 
part  of  academic  coursework.  For  the  purposes  of  this  report,  we  focused 
on  research  institutions  that  conduct  research  as  their  primary  mission 
and  have  dedicated  personnel  to  do  so.  We  excluded  certain  centers  and 
institutes  associated  with  JPME  colleges  and  universities  because  we 
determined  they  did  not  have  a  mission  to  primarily  conduct  research  or 
did  not  have  dedicated  personnel.®  We  also  excluded  from  our  analysis 
research  conducted  by  students  as  part  of  their  academic  coursework 
and  by  professors  not  affiliated  with  the  research  institutions.  To  identify 
the  JPME  research  institutions  included  within  the  scope  of  our  review, 
we  examined  documentation  and  gathered  testimonial  evidence  related  to 
the  missions,  activities,  and  organization  of  the  institutions  that  conduct 
research  at  the  JPME  colleges  and  universities.  Based  on  this  evidence, 
we  determined  that  20  institutions  were  conducting  research  as  their 
primary  mission  and  had  dedicated  personnel  assigned  to  them  and 
therefore  were  included  in  the  scope  of  our  review. 

To  determine  the  extent  to  which  JPME  research  institutions  have 
changed  in  number,  funding,  and  size,  and  the  factors  contributing  to  any 
changes,  we  obtained  questionnaire  responses  and  other  documentation 
on  the  number  of  research  institutions  that  existed  at  JPME  colleges  and 
universities  from  fiscal  years  2000  through  2013  and  collected  and 
analyzed  available  JPME  research  institute  funding  and  staffing  data''®  for 
these  years.  We  assessed  the  reliability  of  the  funding  and  staffing  data 
collected  by  analyzing  questionnaire  responses  from  JPME  colleges  and 
universities,  which  included  information  on  their  data-system 
management,  data  quality-assurance  processes,  and  potential  sources  of 
errors  and  mitigations  of  those  errors.  Based  on  our  review  of  the  data 


®For  example,  although  the  Army  War  College’s  Army  Physical  Fitness  Research  Institute 
conducted  research  and  had  dedicated  personnel,  we  concluded  that  its  resources  were 
primarily  focused  on  educating  and  promoting  the  mental  and  physical  readiness  of 
thousands  of  Army  leaders  and  spouses  each  year.  As  a  result,  we  concluded  that  it  did 
not  meet  both  of  our  criteria  and  is  not  included  within  our  scope. 

“staffing  data  provided  by  JPME  colleges  and  universities  include  military  and  civilian 
personnel,  noncontractor  researchers,  contractor  researchers,  as  well  as  noncontractor 
administrative  staff  positions. 
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provided  and  our  review  of  the  questionnaire  responses,  we  concluded 
that  the  systems  used  to  provide  the  data,  and  thus  the  data  they  provide, 
are  sufficiently  reliable  for  our  audit  purposes.  However,  based  on  this 
evidence,  we  determined  that  we  were  unable  to  report  consistent  data 
on  JPME  research  institution  funding  and  staffing  levels  prior  to  fiscal 
year  2007  and  therefore  we  are  providing  trend  data  on  JPME  research 
institutions  from  fiscal  years  2007  through  2013.'’''  Furthermore,  although 
we  have  identified  a  number  of  factors  that  could  affect  data  quality,  we 
concluded  that  these  were  the  best  available  data  on  JPME  research 
institutions.  We  also  concluded  that  the  data  would  not  lead  to  an 
incorrect  or  unintentional  message  since  they  are  corroborated  through 
interviews  with  DOD  officials.  We  also  discussed  the  reasons  for  any 
trends  in  these  budget  and  staffing  data  with  knowledgeable  officials  in 
DOD  and  at  the  JPME  colleges  and  universities. 

To  determine  the  extent  to  which  DOD  is  assessing  the  performance  of 
JPME  research  institutions  in  meeting  PME  and  other  departmental  goals 
and  objectives,  we  obtained  and  reviewed  documentation  from  the  Joint 
Staff  and  the  JPME  colleges  and  universities  that  identify  any  goals, 
objectives,  or  performance  measures  for  JPME  research  institutions. 
Specifically,  we  reviewed  current  strategic  plans,  mission  statements,  and 
other  documentation  describing  activities  of  the  JPME  research 
institutions.  We  also  reviewed  documentation  describing  external 
oversight  mechanisms  that  monitor  the  academic  quality  of  JPME 
colleges  and  universities.  To  further  our  understanding  of  any  processes 
used  to  assess  the  performance  of  JPME  research  institutions,  we  used  a 
standard  set  of  questions  to  interview  DOD  officials  and  personnel  at  the 
JPME  colleges  and  universities.  We  then  reviewed  the  results  of  the 
interviews  and  related  documents  to  develop  summary  findings.  In 
reviewing  this  documentation  and  testimonial  evidence,  we  referred  to  our 
prior  work  on  best  practices  that  identifies  elements  that  constitute  a 
comprehensive  oversight  framework.  Specifically,  these  best  practices^^ 


^^Officials  at  JPME  colleges  and  universities  stated  that  because  federal  records-retention 
policies  generally  require  agencies  to  maintain  data  for  6  years  and  3  months,  they  were 
unable  to  provide  reliable  data  in  many  cases  prior  to  fiscal  year  2007. 

^^See,  for  example,  GAO,  Building  Partner  Capacity:  Actions  Needed  to  Strengthen  DOD 
Efforts  to  Assess  the  Performance  of  the  Regional  Centers  for  Security  Studies, 

GAO-1 3-606  (Washington,  D.C.:  June  28,  2013)  and  State  Partnership  Program: 
Improved  Oversight,  Guidance,  and  Training  Needed  for  National  Guard’s  Efforts  with 
Foreign  Partners,  GAO-12-548  (Washington,  D.C.:  May  15,  2012). 
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state  that  such  a  framework  should  include  measurable  goals  and 
objectives  linked  with  metrics  for  assessing  progress,  which  is  consistent 
with  the  framework  identified  in  the  Government  Performance  and 
Results  Act  (GPRA),  as  amended  by  the  GPRA  Modernization  Act  of 
2010.''^  We  also  reviewed  this  evidence  in  light  of  key  internal-control 
standards  that  state  that  federal  agencies  should  conduct  reviews  by 
management  at  the  functional  or  activity  level  and  compare  actual 
performance  to  planned  or  expected  results.'''^ 

To  determine  the  extent  to  which  research  activities  of  JPME  research 
institutions  and  other  DOD-funded  research  organizations  are 
coordinated,  we  included  in  the  scope  of  our  review  the  20  JPME 
research  institutions  discussed  above  and  14  other  DOD-funded  research 
organizations.  To  identify  the  other  DOD-funded  research  institutions,  we 
gathered  DOD  documentation  from  the  Assistant  Secretary  of  Defense  for 
Research  and  Engineering  and  canvassed  knowledgeable  DOD  officials 
in  offices  responsible  for  requesting  research,  such  as  the  military 
departments,  science  and  technology  executive  agents,  studies  and 
analysis  research  program  managers,  and  the  Office  of  the  Secretary  of 
Defense,  including  the  Offices  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology  and  Logistics  and  the  Under  Secretary  of 
Defense  for  Policy.  We  reviewed  documentation  from  and  interviewed 
officials  with  these  offices  to  better  understand  the  process  for  requesting 
studies  and  analysis  research  from  JPME  institutions  and  other  DOD- 
funded  research  organizations.  Specifically,  we  used  a  standard  set  of 
questions  to  interview  officials  with  the  Office  of  the  Secretary  of  Defense 
and  the  military  services  about  the  processes  used  to  share  information 
with  other  offices  that  also  request  studies  and  analysis  research.  We 
then  reviewed  the  results  of  the  interviews  and  related  documents  to 
develop  summary  findings.  We  reviewed  the  documentary  and  testimonial 
evidence  in  light  of  key  practices  for  enhancing  and  sustaining 
coordination  as  described  in  our  prior  work  on  best  practices.  Specifically, 
best  practices  state  that  organizations  involved  in  similar  missions  should 
coordinate  and  share  information  to  avoid  unnecessary  duplication  of 


“Pub.  L  No.  103-62  (1993);  Pub.  L  No.  111-352  (2011). 

“GAO,  Standards  for  Internal  Control  In  the  Federal  Government,  GAO/AIMD-00-21.3.1 
(Washington,  D.C.:  November  1999). 
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work.''^  Further,  we  assessed  similarities  and  dissimilarities  among  the 
missions  of  the  JPME  research  institutions  and  other  DOD-funded 
research  organizations  in  the  following  four  ways:  (1 )  we  determined  the 
general  category  of  research  being  conducted  at  JPME  and  other  DOD- 
funded  research  institutions,  for  example,  whether  the  research  was 
studies  and  analysis-  or  science  and  technology-focused;  (2)  we 
reviewed  the  current  mission  statements  for  all  research  organizations 
included  in  our  scope  to  determine  the  topic  areas  in  which  they  conduct 
research;  (3)  we  categorized  research  project  titles  from  2012  and  2013 
provided  to  us  by  all  research  organizations  included  in  our  scope  to 
determine  the  topic  areas  in  which  they  conduct  research;  and  (4)  we 
reviewed  documentation  about  the  offices  that  request  JPME  and  other 
DOD-funded  research  institutions,  along  with  testimonial  evidence 
gathered  during  our  interviews  with  DOD  officials,  to  provide  context  for 
any  similarities  or  dissimilarities  we  identified  through  our  analysis  of 
mission  statements  and  project  titles.  The  results  of  our  analysis  are  not 
generalizable  beyond  the  20  JPME  research  institutions  and  14  other 
DOD-funded  research  institutions  included  in  the  scope  of  our  review.  A 
more-detailed  discussion  of  our  scope  and  methodology  is  included  in 
appendix  I. 

We  conducted  this  performance  audit  from  February  2013  through  March 
2014  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 
that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives. 


^^GAO,  Interagency  Collaboration:  Key  Issues  for  Congressional  Oversight  of  National 
Security  Strategies,  Organizations,  Workforce,  and  Information  Sharing,  GAO-09-904SP 
(Washington,  D.C.:  Sept.  25,  2009). 
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Background 


0v6rvi6W  of  Profossional  The  primary  purpose  of  PME  is  to  develop  military  personnel,  throughout 
Military  Education  their  careers,  for  the  intellectual  demands  of  complex  contingencies  and 

major  conflicts.  The  military  services  provide  PME  at  their  respective  staff 
and  war  colleges.  Each  service  educates  service  members  in  their  core 
competencies  according  to  service  needs.  Air  Force  colleges,  for 
example,  primarily  teach  air  and  space  warfare.  Similarly,  Army,  Navy, 
and  Marine  Corps  colleges  focus  on  land,  maritime,  and  expeditionary 
warfare,  respectively.  DOD  depends  on  the  services’  PME  institutions  to 
develop  personnel  with  these  service-specific  skills.  However,  the  JPME 
program  places  emphasis  on  preparing  leaders  to  conduct  operations  as 
a  coherently  joint  force  in  complex  operating  environments. 

Following  the  passage  of  the  Goldwater-Nichols  Acf®  (the  Act)  in  1986, 
DOD  developed  JPME  as  a  subset  of  learning  within  the  PME  program, 
to  comply  with  the  “joint”  requirements  outlined  in  the  Act  and  subsequent 
legislation.^^  Currently,  JPME  is  provided  at  multiple  sites  across  the 
country,  including  the  services’  staff  and  war  colleges  and  NDU. 

Together,  PME  and  JPME,  prepare  service  members  in  successive 
stages  throughout  their  careers  to  engage  intellectual  challenges 
appropriate  to  increases  in  their  ranks  and  responsibilities.  See  figure  1 
for  a  map  of  service  and  joint  colleges  and  universities  where  JPME  is 
provided. 


^‘^Goldwater-Nichols  Department  of  Defense  Reorganization  Act  of  ^986,  Pub.  L.  No.  99- 
433. 

statutorily  mandated  levels  are  JPME  Phase  I,  which  generally  focuses  on  tactical 
and  operational  levels  of  war;  Phase  II,  which  is  a  follow-on  for  selected  graduates  of 
service  schools  and  other  appropriate  education  programs  that  complements  and 
enhances  the  Phase  I  instruction;  and  the  CAPSTONE  course  of  JPME  for  general/flag 
officers  that  focuses  on  the  operational  and  strategic  levels  of  war  for  high-level  joint, 
interagency,  intergovernmental,  and  multinational  responsibilities.  See  10  U.S.C.  §§  2153- 
2155. 
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Figure  1:  Map  of  Service  and  Joint  Coiieges  and  Universities  Certified  to  Provide  Joint  Professionai  Miiitary  Education  (JPME) 


U  S.  Army  Command  @ 
and  General  Staff  College 
Leavenworth,  Kansas 


r- 


/■ 


Army  War  College  > 
Cartisle,  Pennsylvania  "j. 
@ 

Marine  Corps 
University 

•  U.S.  Marine  Corps 
War  College 

•  U.S.  Marine  Corps 
Command  and 
Staff  College 

Quantico,  Virginia 

© 

Air  University 
•  Aif  War  College 
"^Air  Command 
and  Staff  College 
Montgomery,  Alabama 


•  Naval  War  College 

•  College  of  Naval 
Warfare 

•  College  of  Naval 
Command  and  Staff 

Newport,  Rhode  Island 

•  National  Defense 
University 

•  National  War  College 

•  Eisenhower  School  for 
National  Security  and 
Resource  Management 

•  National  Intelligence 
University 

Washington,  D.C. 

•  Joint  Forces  Staff 
College 

•  Joint  Combined 
Warfighting  School 

•  Joint  Advanced 
Warfighting  School 

•  Advanced  JPME 

Norfolk,  Virginia 


Source  DOO 


Note:  Marine  Corps  University  comprises  two  JPME-granting  colleges — U.S.  Marine  Corps  War 
College  and  U.S.  Marine  Corps  Command  and  Staff  College.  Similarly,  Air  University  comprises  the 
Air  War  College  and  Air  Command  and  Staff  College. 


The  military  services  are  primarily  responsible  for  overseeing  PME  at  their 
respective  staff  and  war  colleges.  As  part  of  their  oversight  efforts,  the 
military  services’  leader-development  efforts  are  included  in  education 
programs.  For  example,  the  Army’s  Training  and  Doctrine  Command 
serves  as  the  executive  agent  for  ensuring  leader  development  is 
integrated  into  PME  courses  at  the  Army  War  College  and  the  Army 
Command  and  Staff  College.  In  contrast,  JPME  is  overseen  by  the  Joint 
Staff.  The  Joint  Staff  is  responsible  for  developing  the  learning  objectives 
for  JPME  and  for  accrediting  the  service  staff  and  war  colleges  and  the 
joint  institutions  to  provide  JPME  coursework.  The  Joint  Staff  also  has 
oversight  responsibility  of  NDU.  The  Chairman  of  the  Joint  Chiefs  of  Staff 
is  statutorily  responsible  for  formulating  policies  for  coordinating  the 
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military  education  and  training  of  members  of  the  armed  forces.''®  The 
Military  Education  Coordination  Council,  which  consists  of  representatives 
from  the  Joint  Staff,  the  service  and  joint  colleges  and  universities,  and 
other  JPME-accredited  institutions,  serves  as  an  advisory  body  to  the 
Joint  Staff  on  joint  education  issues.  The  purpose  of  the  council  is  to 
address  key  educational  issues  of  interest  to  the  joint  educational 
community,  promote  cooperation  and  collaboration  among  the  colleges 
and  universities  certified  to  grant  JPME  degrees,  and  coordinate  joint 
education  initiatives. 

The  Joint  Staff  conducts  periodic  assessments  of  the  three  statutorily 
mandated  levels  of  officer  JPME  to  ensure  that  the  curricula  being  taught 
at  service  staff  and  war  colleges  and  the  joint  institutions  meet  the 
prescribed  joint  educational  requirements  outlined  in  the  Officer 
Professional  Military  Education  Policy.'®  According  to  the  policy,  the 
JPME  program  includes  curriculum  components  that  JPME  colleges  and 
universities  should  follow  to  develop  the  knowledge,  analytical  skills, 
perspectives,  and  values  that  are  essential  for  U.S.  servicemembers  to 
function  effectively  in  joint,  interagency,  intergovernmental,  and 
multinational  operations.  Moreover,  Enclosure  A  of  the  policy  states  that 
senior  officer  studies  at  JPME-degree-granting  colleges  and  universities 
should  emphasize  analysis,  foster  critical  examination,  and  provide 
progressively  broader  educational  experiences. 


DOD  Funds  Various  Types 
of  Research,  Including 
Research  Conducted  at 
JPME  Research 
Institutions 


DOD  funds  several  kinds  of  research.  One  type  of  research  is  studies  and 
analysis  research,  which  DOD  officials  describe  as  analysis  and  advice 
done  to  support  DOD  policy  development,  decision  making,  alternative 
approaches,  and  new  ideas  for  the  DOD  community.  Studies  and  analysis 
research  is  conducted  by  different  organizations  across  DOD.  For 
example,  JPME  research  institutions  can  enhance  strategic  thought  by 
providing,  publishing,  and  disseminating  research  products  for  academic 
coursework,  such  as  on  emerging  threats  students  may  face  in  their 
future  leadership  positions.  To  provide  broad  educational  experiences, 
students  can  conduct  research  at  the  JPME  research  institutions  as  part 
of  their  academic  program.  Students  may  also  work  together  with  the 
research  faculty  resident  at  the  institutions,  providing  them  with  access  to 


^*10  U.S.C.  §  153(a)(5)(c). 
“CJCSI  1 800.01  D. 
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subject-matter  experts  in  their  field  of  study.  Table  1  provides  a  list  of  the 
20  research  institutions  that  are  associated  with  JPME  colleges  and 
universities  and  are  within  the  scope  of  our  review. 


Table  1:  JPME  Colleges  and  Universities  and  Associated  Research  Institutions,  Fiscal  Years  2000  through  2013 


College  or  university  Associated  research  institution 

National  Defense  University  (NDU)  Center  for  Complex  Operations 

Center  for  Strategic  Research 

Center  for  Study  of  Chinese  Military  Affairs 

Center  for  Study  of  Weapons  of  Mass  Destruction 

Center  for  Technology  &  National  Security  Policy 

Conflict  Records  Research  Center 

Center  for  Transatlantic  Security  Studies® 

Air  University  Air  Force  Research  Institute 

Center  for  Strategy  and  Technology 
Air  Force  Counterproliferation  Center 

Naval  War  College  Strategic  Research  Department 

China  Maritime  Studies  Institute 
International  Law  Department 


Strategic  Studies  Group 


Marine  Corps  University 

Flistory  Division 

Middle  East  Studies 

Center  for  Advanced  Operational  Culture  Learning  Translational  Research  Group 

Army  War  College 

Strategic  Studies  Institute 

Army  Command  &  General  Staff  College 

Center  for  Army  Leadership 

Combat  Studies  Institute 

Source:  GAO  analysis  of  DOD  data. 


®NDU’s  Center  for  Transatlantic  Security  Studies  was  disestablished  in  September  2012. 

Other  DOD-funded  organizations  also  conduct  studies  and  analysis 
research.  For  example: 

•  Federally  Funded  Research  and  Development  Centers,  such  as  the 
Center  for  Naval  Analyses  and  the  Institute  for  Defense  Analyses, 
maintain  capabilities  to  conduct  research  in  core  competencies  in 
areas  of  importance  to  DOD,  such  as  analysis,  acquisition  support, 
and  research  and  development.  According  to  a  May  201 1 
memorandum  from  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics,  the  mission  of  Federally  Funded  Research 
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and  Development  Centers  is  to  provide  DOD  with  unique  capabilities 
in  many  areas  where  the  government  cannot  attract  and  retain 
personnel  in  sufficient  depth  and  numbers.  The  memorandum  further 
explains  that  Federally  Funded  Research  and  Development  Centers 
are  a  vital  component  of  the  department’s  overall  acquisition 
workforce  because  they  operate  in  the  public  interest,  free  from 
organizational  conflicts  of  interest. 

•  Service-affiliated  organizations,  such  as  the  Naval  Postgraduate 
School  and  the  Center  for  Army  Analysis,  provide  research  products 
to  their  parent  services  to  help  with  decision  making  and  analysis  on 
critical  issues  facing  the  service. 

•  DOD’s  Regional  Centers  for  Security  Studies  support  DOD’s  objective 
to  build  the  defense  capacity  of  partner  nations. In  our  prior  work, 
we  reported  that  the  Regional  Centers’  activities  include  education, 
exchanges,  research,  and  information  sharing. For  example,  the 
George  C.  Marshall  European  Center  for  Security  Studies  conducts 
research  on  European  security  issues  relevant  to  U.S.  interests. 

JPME  research  institutions  receive  funding  through  their  colleges  and 
universities  and  other  departmental  offices  for  their  operations,  which 
include  research  activities.  Specifically,  most  JPME  colleges  and 
universities  receive  direct  funding  from  their  respective  military  service  to 
fund  their  PME  and  JPME  programs,  and  some  of  those  resources  are 
used  to  fund  their  JPME  research  institutions.  For  example,  the  Naval 
War  College  receives  operation  and  maintenance  and  military  personnel 
funding  from  the  Department  of  the  Navy  as  well  as  funds  in  the  form  of 
monetary  gifts  from  the  Naval  War  College  Foundation.  In  turn,  the 
college  allocates  some  of  those  resources  to  fund  its  associated  research 
institutions,  such  as  the  China  Maritime  Studies  Institute.  However,  NDU 
receives  operation  and  maintenance  funding  from  defense-wide 
appropriations  for  its  JPME  program  and  research  institutions.  With  these 
funds,  JPME  research  institutions  can  support  PME  and  JPME  programs 
as  well  as  the  research  needs  of  those  entities  that  provide  their  funding. 


^°DOD’s  Regional  Centers  for  Security  Studies  include  the  George  C.  Marshall  European 
Center  for  Security  Studies,  the  Near  East  South  Asia  Center  for  Strategic  Studies,  the 
William  J.  Perry  Center  for  Western  Hemisphere  Defense  Studies,  the  Africa  Center  for 
Strategic  Studies,  and  the  Asia-Pacific  Center  for  Security  Studies. 

"^GAO-1 3-606. 
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For  example,  the  Air  University’s  Center  for  Strategy  and  Technology 
produces  research  that  is  responding  to  key  questions  and  topics  of 
interest  posed  by  the  Chief  of  Staff  of  the  Air  Force. 

Additionally,  some  JPME  research  institutions  receive  funding  on  a 
reimbursable  basis  from  other  departmental  offices,  such  as  the 
directorates  within  the  Office  of  the  Secretary  of  Defense,  the  Joint  Staff, 
and  the  military  services. These  offices  receive  their  own  funding,  such 
as  research,  development,  test,  and  evaluation  funds  and  operation  and 
maintenance  funds,  which  may  be  used  in  part  to  fund  annual 
requirements  for  research  projects.  To  fulfill  these  annual  research 
requirements,  funding  may  be  allocated  to  JPME  research  institutions. 
Federally  Funded  Research  and  Development  Centers,  or  think  tanks^^  to 
conduct  individual  research  projects  in  support  of  those  offices’  annual 
research  requirements.  For  example,  NDU’s  research  institutions  have 
received  funding  from  the  Office  of  the  Secretary  of  Defense  to  study  and 
build  subject-matter  expertise  on  issues  related  to  Afghanistan  and 
Pakistan. 

DOD  also  funds  science  and  technology-related  research.  According  to 
testimony  in  April  2013  by  Acting  Assistant  Secretary  of  Defense  for 
Defense  Research  &  Engineering,  science  and  technology  research  is 
funded  to  mitigate  new  or  emerging  capabilities  that  could  degrade  U.S. 
capabilities,  enable  new  or  extended  capabilities  in  existing  military 
systems,  and  develop  new  concepts  and  technologies  through  science 
and  engineering  applications  to  military  problems. Science  and 
technology  research  is  conducted  under  the  auspices  of  the  Office  of  the 
Assistant  Secretary  of  Defense  for  Research  and  Engineering.  According 
to  DOD  guidance,  this  office  develops  the  strategies  and  supporting  plans 
for  utilizing  technology  to  respond  to  DOD  needs  and  ensures  U.S. 


^^For  the  purposes  of  this  report,  reimbursable  funding  refers  to  amounts  earned  or 
collected  from  outside  organizations  for  research  services  furnished  by  the  institution. 

^^For  the  purposes  of  this  report,  a  think  tank  is  defined  as  a  nonprofit  organization  that 
conducts  public  policy  research  and  analysis.  An  example  of  a  think  tank  is  the  Brookings 
Institution,  based  in  Washington,  D.C. 

^^Alan  R.  Shaffer,  Acting  Assistant  Secretary  of  Defense  for  Defense  Research  & 
Engineering,  Budget  Request  for  DOD  Science  and  Technology  Programs,  testimony 
before  the  Subcommittee  on  Intelligence,  Emerging  Threats  and  Capabilities,  Committee 
on  Armed  Services,  Flouse  of  Representatives,  11 3th  Cong.,  1st  sess.,  April  16,2013. 
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technological  superiority.^®  The  office  is  the  executive  secretary  for  DOD’s 
Research  and  Engineering  Executive  Committee.  This  committee  brings 
together  leadership  from  the  DOD  components  that  have  science  and 
technology  research  investments  for  the  purpose  of  strengthening 
coordination  and  enhancing  the  efficiency  of  research  and  engineering 
investments  in  areas  that  cannot  be  addressed  adequately  by  any  single 
component.  Science  and  technology  research  comprises  basic  research, 
applied  research,  and  advanced  technology  development. JPME 
research  institutions  do  not  conduct  science  and  technology  research. 
Science  and  technology  research  is  generally  conducted  by  DOD 
laboratories  associated  with  the  military  services,  such  as  the  Army 
Research  Laboratory,  and  some  Federally  Funded  Research  and 
Development  Centers,  such  as  the  Massachusetts  Institute  of  Technology 
Lincoln  Laboratory,  among  other  organizations. 


JPME  Research 
Institutions  Expanded 
from  2007  through 
2011  Due  to  Several 
Factors,  but  Have 
Subsequently 
Contracted 


JPME  research  institutions,  particularly  at  NDU,  experienced 
considerable  growth  in  number,  funding,  and  size  in  terms  of  staffing 
levels  from  fiscal  year  2007  through  fiscal  year  201 1  but  have  declined 
over  the  past  2  years.  Several  factors  contributed  to  JPME  research 
institution  growth,  including  increases  in  reimbursable  funding  from 
outside  offices  sponsoring  JPME  research,  the  creation  of  new  research 
institutions,  and  the  realignment  of  institutions  at  some  JPME  colleges 
and  universities.  While  a  variety  of  factors  contributed  to  the  expansion  of 
JPME  research  institutions,  it  has  primarily  been  department-wide  budget 
reductions  that  contributed  to  their  decreases  in  number,  funding,  and 
size  since  2011. 


^^Department  of  Defense,  Director  of  Defense  Research  and  Engineering,  Directive 
5134.3  (Nov.  3,  2003). 

^“^Basic  research  is  the  systematic  study  directed  toward  greater  knowledge  or 
understanding  of  the  fundamental  aspects  of  phenomena  and  of  observable  facts  without 
specific  applications  toward  processes  or  products  in  mind.  Applied  research  is  the 
systematic  study  to  gain  knowledge  or  understanding  necessary  to  determine  the  means 
by  which  a  recognized  and  specific  need  may  be  met.  Advanced  technology  development 
includes  all  efforts  that  have  moved  into  the  development  and  integration  of  hardware  for 
field  experiments  and  tests. 
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Number,  Funding,  and 
Size  of  JPME  Research 
Institutions  Grew  from 
2007  through  2011,  Then 
Declined 


Number  of  JPME  Research 
Institutions 


JPME  research  institutions,  particularly  at  NDU,  experienced 
considerable  growth  in  number,  funding,  and  size  in  terms  of  staffing 
levels  from  fiscal  year  2007  through  fiscal  year  201 1 ,  but  have  declined 
over  the  past  2  years.  The  following  sections  discuss  overall  trends  in  the 
number  of  research  institutions  at  JPME  colleges  and  universities  from 
fiscal  year  2007  through  fiscal  year  2013,  as  well  as  overall  trends  in 
funding  and  staffing  levels  for  this  same  period. Appendix  II  provides 
more-detailed  information  for  each  of  the  20  JPME  research  institutions 
from  fiscal  year  2004  through  fiscal  year  2013,  as  available. 

From  2007  through  201 1 ,  the  number  of  JPME  research  institutions  grew 
from  14  to  20.  During  this  period,  the  number  of  research  institutions  at 
NDU  increased  by  3.  At  Marine  Corps  University  and  Air  University,  the 
number  of  research  institutions  increased  by  2  and  1 ,  respectively.  Since 
2011,  however,  the  number  of  research  institutions  has  slightly  declined 
due  to  the  disestablishment  of  the  Center  for  Transatlantic  Security 
Studies  at  NDU  in  2012.  Figure  2  shows  the  total  number  of  JPME 
research  institutions  for  fiscal  years  2007  through  2013. 


^'^Officials  at  PME  colleges  and  universities  were  unable  to  provide  consistent  data  on 
JPME  research  institution  funding  and  staffing  levels  prior  to  fiscal  year  2007.  Therefore, 
we  are  providing  composite  data  on  JPME  research  institutions  from  fiscal  year  2007 
through  fiscal  year  201 3. 


Page  14 


GAO-14-216  JPME  Research  Institutions 


Funding  for  JPME  Research 
Institutions 


Figure  2:  Number  of  Joint  Professionai  Miiitary  Education  (JPME)  Research 
Institutions,  Fiscal  Years  2007  through  2013 


Number 


2007  2008  2009  2010  2011  2012  2013 

Fiscal  year 


Source:  GAO  analysis  of  DOD  data. 


Funding  for  research  institutions  at  JPME  colleges  and  universities 
experienced  growth  from  fiscal  year  2007  through  2011.  Specifically,  total 
funding  for  JPME  research  institutions  increased  from  about  $31.0  million 
in  fiscal  year  2007  to  about  $47.7  million  in  fiscal  year  201 1 .  Much  of  the 
growth  took  place  at  NDU,  where  research  institutions’  total  funding 
increased  by  about  78  percent.  Other  JPME  colleges  and  universities 
also  experienced  considerable  increases  in  funding  for  the  operation  of 
their  associated  research  institutions.  For  example,  with  the 
establishment  of  the  Middle  East  Studies  institute  and  the  Center  for 
Advanced  Operational  Culture  Learning’s  Translational  Research  Group 
in  2007  and  2010,  respectively,  funding  for  Marine  Corps  University’s 
research  institutions  increased  from  $156,000  in  fiscal  year  2007  to  about 
$4.9  million  in  fiscal  year  201 1 . 

Since  fiscal  year  201 1 ,  funding  for  JPME  research  institutions  decreased 
overall.  Specifically,  total  funding  for  JPME  research  institutions  fell  by 
about  1 5  percent  from  fiscal  year  201 1  through  2013,  from  about  $47.7 
million  to  about  $40.6  million.  Much  of  the  decline  reflects  decreases  at 
NDU,  where  research  institutions  experienced  a  21  percent  decrease  in 
total  funding  from  about  $21 .4  million  in  fiscal  year  2011  to  about  $1 6.8 
million  in  fiscal  year  2013.  The  Army  Command  and  General  Staff 
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College’s  Combat  Studies  Institute  and  the  Center  for  Army  Leadership 
also  experienced  considerable  declines  during  this  period,  as  total 
funding  for  both  decreased  by  about  19  percent.  Figure  3  provides  total 
funding  for  JPME  research  institutions  by  JPME  college  and  university  for 
fiscal  years  2007  through  2013. 


Figure  3:  Total  Funding  for  Joint  Professional  Military  Education  (JPME)  Research 
Institutions  by  JPME  College  and  University,  Fiscal  Years  2007  through  2013 
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Source:  GAO  analysis  of  DOD  data. 

Note:  Nominal  dollars,  unadjusted  for  inflation. 
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staffing  Levels  at  JPME 
Research  Institutions 


Staffing  levels  at  JPME  research  institutions  also  increased  considerably 
from  fiscal  year  2007  through  fiscal  year  201 1 .  Specifically,  staffing  levels, 
in  terms  of  full-time  equivalents,  increased  from  207  in  fiscal  year  2007  to 
384  in  fiscal  year  201 1 ,  about  an  86  percent  increase.^®  In  particular,  total 
staffing  levels  at  NDU’s  research  institutions  increased  by  about  58 
percent  during  this  period  while  other  JPME  colleges  and  universities  also 
experienced  growth  in  staffing  levels.  For  example,  total  staffing  levels  at 
Air  University’s  research  institutions  increased  from  19  to  97. 

Since  201 1 ,  total  staffing  levels  at  JPME  research  institutions  decreased 
from  384  to  310,  about  a  19  percent  decrease.  Much  of  the  decrease  is 
the  result  of  a  decline  in  staffing  levels  at  NDU,  where  research 
institutions  experienced  a  31  percent  decline  during  this  period.  Figure  4 
shows  staffing  levels  for  JPME  research  institutions  by  JPME  college  and 
university  for  fiscal  years  2007  through  2013. 


^“DOD  reported  staffing  level  data  in  full-time  equivalents,  which  are  calculated  as  the  total 
hours  worked  in  jobs  retained  divided  by  the  number  of  hours  in  a  full  time  schedule,  and 
include  noncontractor  researchers,  contractor  researchers,  and  noncontractor 
administrative  staff  positions. 
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Figure  4:  Staffing  Leveis  for  Joint  Professionai  Miiitary  Education  (JPME)  Research 
Institutions  by  JPME  College  and  University,  Fiscal  Years  2007  through  2013 
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Several  Factors 
Contributed  to  JPME 
Research  Institution 
Expansion  and 
Subsequent  Contraction 


Several  factors  contributed  to  JPME  research  institution  growth  from 
fiscal  year  2007  through  fiscal  year  201 1 ,  including  increases  in 
reimbursable  funding  provided  by  outside  offices  sponsoring  JPME 
research,  the  creation  of  new  research  institutions,  the  realignment  of 
institutions  such  that  they  were  incorporated  into  JPME  colleges  and 
universities,  and  an  increase  in  resources  dedicated  to  research  at  some 
JPME  colleges  and  universities.  According  to  DOD  officials,  these 
increases  occurred  within  the  context  of  the  then-ongoing  operations  in 
Iraq  and  Afghanistan,  for  which  the  Joint  Staff  and  the  military  services 
expected  the  JPME  research  institutions  to  provide  increased  support  to 
the  warfighter.  In  particular,  these  factors  led  to  an  expansion  at  NDU’s 
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research  institutions  during  this  period.  For  example,  NDU’s  research 
budget  grew  primarily  due  to  increases  in  reimbursable  research  funded 
by  outside  offices  such  as  the  Office  of  the  Secretary  of  Defense  and  the 
Joint  Staff.  Reimbursable  funding  provided  to  NDU’s  research  institutions 
increased  from  about  $5.6  million  in  2007  to  about  $14.6  million  in  fiscal 
year  2011,  as  shown  in  figure  5.  Specifically,  reimbursable  funding  for 
NDU’s  Center  for  Technology  and  National  Security  Policy’s  research 
increased  from  about  $3.8  million  in  fiscal  year  2007  to  about  $6.9  million 
in  fiscal  year  201 1 .  Additionally,  funding  for  NDU’s  Center  for  Study  of 
Weapons  of  Mass  Destruction,  which  is  funded  entirely  on  a  reimbursable 
basis,  more  than  doubled  from  $1.5  million  in  fiscal  year  2007  to  $3.1 
million  in  fiscal  year  201 1 . 


Figure  5:  Total  Reimbursable  Funding  for  the  National  Defense  University’s  (NDU) 
Research  Institutions,  Fiscal  Years  2007  through  2013 
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Source.  GAO  analysis  of  DOD  data. 

Note:  Nominal  dollar,  unadjusted  for  inflation. 


NDU  officials  attributed  the  growth  in  reimbursable  funding  to  the  fact  that 
research  had  evolved  into  a  key  area  of  emphasis  for  the  university. 
These  officials  also  noted  that,  as  a  result,  faculty  members  at  the 
research  institutions  were  encouraged  to  pursue  research  directly  funded 
by  other  DCD  entities  and  other  U.S.  government  agencies.  To  meet  the 
increased  demands  for  reimbursable  research,  particularly  when  in-house 
expertise  did  not  exist,  NDU  increased  the  number  of  contractor  and 
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noncontractor  researchers  at  its  research  institutions.^®  Joint  Staff  officials 
noted  that  because  a  significant  portion  of  the  NDU  workforce,  including 
that  of  its  research  institutions,  is  made  up  of  senior-level  positions, 
researcher  salaries  contributed  to  the  growth  in  NDU’s  research  budget. 

We  also  found  that  funding  for  NDU’s  research  institutions  increased  as 
the  result  of  the  transfer  of  research  institutions  to  NDU  as  well  as  the 
broadening  of  missions  of  other  research  institutions.  For  example, 
according  to  DOD  officials,  in  an  effort  to  better  deliver  education  to  DOD 
and  other  U.S.  government  personnel  on  issues  related  to  ongoing 
operations  in  Iraq  and  Afghanistan,  such  as  irregular  warfare, 
counterinsurgency,  and  stability  and  reconstruction  operations,  the  Center 
for  Complex  Operations  was  transferred  from  the  Defense  Security 
Cooperation  Agency  to  NDU  in  2009.  Furthermore,  funding  for  the  Center 
for  the  Study  of  Weapons  of  Mass  Destruction  at  NDU  increased  due  to 
DOD’s  decision  to  broaden  the  Center’s  counterproliferation  focus 
government-wide.  Moreover,  the  Chairman  of  the  Joint  Chiefs  of  Staff 
designated  the  center  as  the  focal  point  for  weapons  of  mass  destruction 
education  for  JPME.  As  a  result,  the  Joint  Staff  began  to  fund  the  center 
in  2008  to  perform  that  mission. 

Other  JPME  colleges  and  universities  also  experienced  considerable 
increases  in  research  institution  funding  and  staffing  levels  from  fiscal 
year  2007  through  fiscal  year  201 1  due  to  factors  such  as  the  creation  of 
new  research  institutions  and  the  realignment  of  others  within  JPME 
colleges  and  universities.  For  instance,  according  to  Air  Force 
documentation  and  officials.  Air  University’s  increase  in  staffing  levels  for 
its  research  institutions  can  largely  be  attributed  to  the  establishment  of 
the  Air  Force  Research  Institute  in  2008.  According  to  an  Air  University 
official,  the  establishment  of  the  Air  Force  Research  Institute  resulted 
from  the  consolidation  and  realignment  of  personnel  from  existing  Air 
Force  institutions  as  well  as  the  creation  of  18  new  positions.  Air  Force 
officials  also  stated  that  staffing  levels  at  the  Air  Force 
Counterproliferation  Center  and  the  Air  Force  Center  for  Strategy  and 
Technology  increased  due  to  increased  research  requests  from  the  Air 
Staff  on  nuclear  and  strategic-level  research  projects,  respectively. 

Marine  Corps  University’s  increase  in  staffing  levels  at  its  three  research 


According  to  an  NDU  official,  the  vast  majority  of  NDU’s  noncontractor  researchers  serve 
on  renewable  term  appointments  ranging  from  1  to  3  years. 
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institutions  can  be  attributed  to  the  establishment  of  the  Translational 
Research  Group  in  2010  and  increases  in  staff  positions  within  the 
History  Division  and  Middle  East  Studies  center. 

While  a  variety  of  factors  contributed  to  the  expansion  of  JPME  research 
institutions,  it  has  primarily  been  department-wide  budget  reductions, 
including  the  implementation  of  sequestration  in  fiscal  year  2013,  that 
contributed  to  their  decreases  in  number,  funding,  and  size.  For  example, 
officials  stated  that  decreases  in  funding  for  NDU’s  research  institutions 
and  staffing  levels  resulted  from  overall  reductions  at  the  university  due  to 
declining  budgets.  Furthermore,  according  to  officials,  NDU’s  budget  for 
its  research  institutions  came  under  increased  scrutiny  in  2011  with  the 
issuance  of  a  new  mission  statement  for  NDU  by  the  Chairman  of  the 
Joint  Chiefs  of  Staff  that  prioritized  research  that  more-directly  supported 
education  over  reimbursable  research.  Moreover,  according  to  officials, 
the  Joint  Staff  established  full-time  equivalent  caps  for  both  direct  and 
reimbursable  funding  in  2012.  As  a  result  of  actions  taken  to  reduce 
NDU’s  budget,  NDU’s  Center  for  Transatlantic  Studies  was  disestablished 
in  September  2012,  and  the  decision  was  made  in  early  2013  to  defund 
the  Conflict  Records  Research  Center  beginning  in  fiscal  year  2014. 

Similarly,  DOD-wide  budget  reductions  contributed  to  decreases  in  the 
funding  and  size  of  other  JPME  research  institutions.  For  example.  Naval 
War  College  officials  stated  that  its  research  institutions  absorbed  a 
majority  of  the  college’s  budget  cuts  since  fiscal  year  201 1  because  the 
college  prioritized  funds  to  support  its  principal  education  mission.  Air 
University  has  also  experienced  decreasing  budgets  and  staffing  level 
reductions  beginning  in  fiscal  year  201 1  through  fiscal  year  201 3. 
Specifically,  we  found  that  the  Air  Force  Research  Institute’s  total  staffing 
levels  decreased  from  a  high  of  81  in  fiscal  year  201 1  to  61  in  fiscal  year 
2013  as  a  result  of  overall  reductions  in  Air  Force  civilian  personnel. 
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DOD  Is  Limited  in  Its 
Ability  to  Assess  the 
Performance  of  JPME 
Research  Institutions 


The  extent  to  which  DOD  can  assess  the  performance  of  JPME  research 
institutions  is  limited  by  the  lack  of  a  comprehensive  framework  to 
systematically  assess  their  performance  in  meeting  PME  and  other 
departmental  goals  and  objectives.  With  limited  exceptions,  the  JPME 
colleges  and  universities,  which  have  broad  latitude  in  overseeing  their 
associated  research  institutions,  have  not  consistently  established 
measurable  goals  or  objectives  linked  with  metrics  to  assess  the 
performance  of  their  associated  research  institutions.  However,  best 
practices^”  state  that  achieving  results  in  government  requires  a 
comprehensive  framework  that  includes  measurable  goals  and  objectives 
and  metrics  for  assess  progress,  consistent  with  the  framework  identified 
in  the  Government  Performance  and  Results  Act.^''  Further,  while  there 
are  mechanisms  in  place  for  overseeing  JPME  colleges  and  universities, 
such  as  the  Joint  Staff’s  JPME  accreditation  process,  these  are  focused 
on  the  quality  of  academic  programs  and  not  on  the  research  institutions’ 
performance.  There  is  no  DOD-wide  guidance  that  addresses  the 
intended  role  of  the  research  institutions  in  supporting  PME  or  other 
departmental  goals  or  assigns  responsibilities  for  conducting  reviews  of 
them,  leaving  the  department  without  a  basis  to  assess  the  institutions’ 
stated  mission  and  actual  performance  against  planned  or  expected 
results.  This  is  inconsistent  with  the  Standards  for  Internal  Control  in  the 
Federal  Government,  which  state  that  agencies  should  conduct  reviews 
by  management  at  the  functional  or  activity  level,  which  in  this  case  would 
be  the  JPME  research  institutions,  and  compare  actual  performance  to 
planned  or  expected  results. Clearly  establishing  linkages  between 
significant  activities,  their  intended  role  in  meeting  agency-wide  goals  and 
objectives,  and  assigning  oversight  responsibilities  underpins  an  agency’s 
ability  to  conduct  such  reviews. 


DOD  Does  Not  Have  a 
Framework  to  Assess  the 
Performance  of  JPME 
Research  Institutions 


According  to  officials  representing  the  Joint  Staff  and  JPME  colleges  and 
universities,  DOD  has  provided  JPME  colleges  and  universities  with 
broad  latitude  in  overseeing  their  associated  research  institutions.  In 
doing  so,  the  Joint  Staff  and  the  military  services  have  not  provided 
guidance  to  assist  the  JPME  colleges  and  universities  in  developing  a 


^°See,  for  example,  GAO-13-606  and  GAO-12-548. 

"'Pub.  L  No.  103-62  (1993),  as  amended  by  Pub.  L  No.  111-352  (2011). 
""GAO/AIMD-00-21.3.1. 
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comprehensive  oversight  framework  for  assessing  the  performance  of 
JPME  research  institutions.  As  a  result,  we  found  that  JPME  colleges  and 
universities  have  not  consistently  established  measurable  goals  and 
objectives  linked  with  performance  metrics  to  assess  the  performance  of 
their  associated  research  institutions  and  therefore  are  unable  to 
comprehensively  assess  their  performance  to  determine  whether  they  are 
furthering  JPME  or  other  departmental  goals.  According  to  best 
practices, achieving  results  in  government  requires  a  comprehensive 
oversight  framework  that  includes  measurable  goals  and  objectives,  and 
metrics  for  assessing  progress,  consistent  with  the  framework  identified  in 
the  Government  Performance  and  Results  Act. 

Based  on  our  review  of  available  strategic  plans,  mission  statements,  and 
other  documentation,  such  as  strategic  issues  lists,  that  has  guided  the 
work  of  the  20  JPME  research  institutions,  we  found  that  JPME  colleges 
and  universities  have  not  consistently  established  measurable  goals  and 
objectives  linked  with  performance  metrics  for  their  associated  research 
institutions.  Specifically,  we  identified  limited  instances  where  specific 
goals  or  objectives  for  associated  JPME  research  institutions  have  been 
established.  Air  University’s  strategic  plan,  for  example,  establishes  a 
goal  for  the  Air  Force  Research  Institute  to  produce  and  publish  scholarly, 
credible,  relevant,  and  useful  research  and  outreach  products.  This  goal 
is  consistent  with  best  practices  of  being  measurable  and  specific. 
Recently,  NDU  has  taken  steps  intended  to  provide  a  basis  for  assessing 
the  performance  of  the  university’s  research  institutions.  Specifically,  in 
2013  following  a  review  by  the  Joint  Staff, NDU  developed  a  strategic 
plan  for  research  that  establishes  linkages  between  the  university’s 
strategic  goals  and  objectives  for  NDU’s  research  institutions.  For 
example,  one  of  the  research  goals  is  to  address  the  knowledge  and  skills 
for  understanding  and  leading  in  a  rapidly  changing  global  security 
environment,  which  reflects  one  of  NDU’s  strategic  goals.  In  order  to  carry 
out  this  objective,  NDU’s  plan  calls  for  research  faculty  to  engage  with 


^^See,  for  example,  GAO-13-606  and  GAO-12-548. 

April  2012,  the  Joint  Staff  conducted  a  management  control  review  of  NDU,  the 
purpose  of  which  was  to  assess  the  administrative  and  fiscal  control  processes  that  were 
in  place  to  ensure  proper  stewardship  of  NDU’s  resources.  As  part  of  that  review,  the  Joint 
Staff  noted  that  throughout  NDU  there  appeared  to  be  a  fundamental  disagreement 
regarding  how  its  research  supported  the  JPME  mission  and  courses.  Accordingly,  the 
Chairman  of  the  Joint  Chiefs  of  Staff  provided  NDU  with  a  new  mission  statement 
emphasizing  that  its  research  should  support  its  academic  mission. 
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teaching  faculty  to  bring  together  expertise  in  national  security  studies. 
The  NDU  President  approved  the  strategic  plan  for  research  in  January 
2014. 

We  identified  other  examples  where  JPME  colleges  and  universities 
identified  broad  goals  and  objectives  for  research.  However,  the  linkage 
between  these  goals  and  objectives  and  JPME  research  institutions  was 
unclear  as  the  goals  and  objectives  are  not  specifically  assigned  to 
associated  JPME  research  institutions.  For  example,  the  Naval  War 
College’s  strategic  plan  contains  a  guiding  principle  to  keep  the  college’s 
research  and  scholarly  activities  relevant  to  the  needs  of  the  Navy  and 
the  nation.  Similarly,  Marine  Corps  University’s  strategic  plan  contains  a 
goal  related  to  strengthening  professional  scholarship  and  outreach. 
However,  neither  of  these  goals  makes  reference  to  the  college’s  or 
university’s  research  institutions.  Additionally,  JPME  colleges  and 
universities,  such  as  Air  University  and  Army  War  College,  have 
developed  lists  of  research  priorities  on  an  annual  basis.  According  to 
officials,  these  lists  are  developed  to  reflect  the  priorities  of  senior 
leadership  within  their  service  and  have  been  used  to  guide  the  research 
activities  of  JPME  students.  In  2013,  the  Army  War  College  completed  a 
strategic  review  of  its  academic  programs  and,  as  a  result  of  this  review, 
has  aligned  the  development  of  its  Key  Strategic  Issues  List  with  a 
specific  strategic  goal  of  influencing  national  security  decision-making. 
However,  the  Army  War  College  has  not  clearly  linked  its  strategic  issues 
list  with  the  education  goals  of  the  college. 

Furthermore,  we  found  that  JPME  colleges  and  universities  have  not 
consistently  established  metrics  to  assess  the  performance  of  JPME 
research  institutions  in  meeting  PME  or  other  departmental  needs.  Based 
on  our  review,  we  identified  some  examples  where  JPME  colleges  and 
universities  had  established  performance  metrics  for  their  associated 
research  institutions.  For  example.  Air  University  established  a 
performance  measure  for  the  Air  Force  Research  Institute  that  includes  a 
count  of  the  requested  versus  the  delivered  research  studies  for  senior 
Air  Force  staff.  Similarly,  Marine  Corps  University  established  several 
measures  to  assess  the  progress  its  research  institutions  have  made  in 
achieving  desired  outcomes.  For  example,  the  university  established  a 
measure  for  the  History  Division  intended  to  assess  its  responsiveness  to 
research  inquiries. 

Officials  from  JPME  colleges  and  universities,  including  JPME  research 
institutions,  told  us  that  they  recognize  the  need  to  establish  measures  for 
assessing  the  research  institutions’  performance.  They  explained. 
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however,  that  they  have  faced  difficulties  in  developing  them  for  research 
institutions.  For  example,  officials  representing  JPME  colleges  and 
universities  stated  that  it  is  challenging  to  compile  quantitative  data  that 
represent  the  value  or  the  usefulness  of  research.  Although  we  recognize 
that  it  is  difficult  to  establish  performance  measures  for  outcomes  that  are 
not  readily  observable  or  in  some  cases  systematic, the  department 
does  use  metrics  to  assess  the  performance  of  other  DOD-funded 
organizations  that  conduct  studies  and  analysis  research.  For  example, 
DOD  guidance  directs  organizations  that  sponsor  a  Federally  Funded 
Research  and  Development  Center  to  assess  their  performance.^® 
According  to  the  guidance,  sponsoring  organizations  must  develop 
procedures  to  annually  monitor  the  value,  quality  and  responsiveness  of 
their  work.  For  instance,  officials  we  spoke  with  within  the  Office  of  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics 
stated  that  their  office  compiles  data  on  metrics  for  the  RAND  National 
Defense  Research  Institute — a  Federally  Funded  Research  and 
Development  Center — that  are  primarily  based  upon  quantitative  ratings 
and  comments  gathered  from  surveying  organizations  that  contracted  for 
research  projects  with  it.  For  those  research  projects  that  received  low 
ratings  or  negative  comments,  the  RAND  National  Defense  Research 
Institute  is  required  to  follow  up  with  sponsors  to  understand  what  had 
happened  and  provide  a  plan  for  corrective  actions. 

Without  a  framework  that  includes  measurable  goals  and  objectives 
linked  with  metrics,  DOD,  including  its  JPME  colleges  and  universities, 
does  not  have  a  systematic  basis  to  comprehensively  assess  the 
performance  of  JPME  research  institutions.  Best  practices  state  that  a 
framework  that  consists  of  measurable  goals  and  objectives  linked  with 
metrics  for  assessing  progress  would  better  enable  DOD  to  determine 
whether  JPME  research  institutions  are  achieving  results.  Moreover,  it 
would  provide  the  DOD  with  a  sounder  basis  for  making  resource 
determinations  to  ensure  that  these  research  institutions  are  furthering 
JPME  and  other  departmental  goals  and  that  JPME  continues  to  provide 
servicemembers  with  the  expertise  necessary  for  their  careers. 


^^See,  for  example,  GAO-13-606  and  GAO-12-548. 

^®Department  of  Defense,  Under  Secretary  of  Defense  for  Acquisition,  Technology  and 
Logistics,  Federally  Funded  Research  and  Development  Center  (FFRDC)  Management 
Plan  and  Associated  “How-to-Guldes,  ”  Memorandum  (May  2,  201 1 ). 
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Established  Oversight 
Processes  Do  Not  Assess 
the  Performance  of  JPME 
Research  Institutions 


Standards  for  Internal  Control  in  the  Federal  Government  state  that 
agencies  should  conduct  reviews  by  management  at  the  functional  or 
activity  level  and  compare  actual  performance  to  planned  or  expected 
results. Clearly  establishing  linkages  between  significant  activities,  their 
intended  role  in  helping  meet  agency-wide  goals  and  objectives,  and 
assigning  oversight  responsibilities  underpins  an  agency’s  ability  to 
conduct  such  reviews.  According  to  these  standards,  such  controls  are  an 
integral  part  of  an  agency’s  planning,  implementing,  reviewing,  and 
accountability  for  stewardship  of  government  resources  and  achieving 
effective  results.  However,  the  oversight  conducted  by  the  Joint  Staff  and 
by  external  accrediting  bodies  reviews  the  quality  of  JPME  academic 
programs  and  not  the  JPME  research  institutions’  performance.  For 
example,  the  Joint  Staff’s  Process  for  Accreditation  of  Joint  Education 
process  is  DOD’s  primary  mechanism  of  oversight,  assessment,  and 
improvement  of  JPME  academic  programs.  The  instruction  governing  this 
process^®  lays  out  seven  educational  standards  common  to  all  PME 
colleges  and  universities,  including  JPME  colleges  and  universities,  which 
the  Chairman  of  the  Joint  Chiefs  of  Staff  considers  essential.®®  Officials 
from  the  Joint  Staff  JPME  Division  stated  that  this  office  conducts  reviews 
of  the  JPME  colleges  and  universities  every  6  years  to  determine  how 
well  their  academic  programs  are  meeting  these  education  standards. 


Although  the  Chairman  of  the  Joint  Chiefs  of  Staff  is  statutorily 
responsible  for  overseeing  the  officer  joint  education  system,  the  Joint 
Staff  instruction  that  serves  as  the  primary  guidance  for  JPME-related 
policy  does  not  address  what  the  intended  role  of  research  institutions 
should  be  at  JPME  colleges  and  universities  and  it  does  not  assign 
responsibilities  for  conducting  oversight  of  their  activities.  Without  clarity 
as  to  the  intended  role  of  research  institutions  in  support  of  JPME 
academic  programs  or  for  another  purpose,  there  is  no  basis  by  which  to 
compare  the  research  institutions’  respective  stated  missions  as  well  as 


""GAO/AIMD-OO-21.3.1. 

"“CJCSI  1 800.01  D. 

According  to  Enclosure  E  of  CJCSI  1 800.01  D,  the  seven  educational  standards  are  (1) 
develop  joint  awareness,  perspective,  and  attitudes;  (2)  employ  predominantly  active  and 
highly  effective  instruction  methods;  (3)  assess  student  achievement;  (4)  assess  program 
effectiveness;  (5)  conduct  quality  faculty  recruitment;  selection,  assignment,  and 
performance  assessment  program;  (6)  conduct  faculty  development  programs  for 
improving  instructional  skills  and  increasing  subject  matter  mastery;  and  (7)  provide 
institutional  resources  to  support  the  educational  process. 
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actual  performance  to  planned  or  expected  results,  as  required  by 
Standards  for  Internal  Control/®  Specifically,  the  instruction  contains 
standards  to  evaluate  the  quality  of  JPME  academic  programs,  but  it  does 
not  define  the  role  of  JPME  research  institutions  and  it  contains  no 
educational  standards,  learning  areas,  or  specific  objectives  for  research 
that  would  enable  DOD  to  assess  the  performance  of  research  institutions 
at  JPME  colleges  and  universities/''  Further,  no  organization  is  assigned 
specific  responsibility  for  overseeing  the  performance  of  the  research 
institutions. 

Joint  Staff  officials  agreed  that  their  accreditation  process  is  focused 
specifically  on  JPME  academic  programs  and  pertains  only  to  academic 
curricula  development  and  quality  and  the  assurance  of  uniformity  in 
course  content  across  the  different  JPME  colleges  and  universities.  As  a 
result,  JPME  research  institutions  are  not  reviewed  as  part  of  the  Joint 
Staff’s  accreditation  process.  Moreover,  according  to  Joint  Staff  officials, 
while  they  are  statutorily  responsible  for  overseeing  the  quality  of  JPME 
academic  programs,  there  is  no  statutory  responsibility  for  the  Joint  Staff 
to  oversee  the  performance  of  JPME  research  institutions.  One  JPME 
official  with  whom  we  met  noted  that  if  the  department  had  goals, 
objectives,  and  performance  measures  for  JPME  research  institutions  as 
part  of  the  instruction  governing  the  Joint  Staff  accreditation  process,  it 
would  strengthen  the  department’s  oversight  process  for  JPME  research 
institutions.  In  the  absence  of  DOD-wide  guidance  that  defines  the  role  of 
research  institutions  as  part  of  the  JPME  system  and  establishes  roles 
and  responsibilities  for  conducting  oversight  of  JPME  research 
institutions,  the  department  and  JPME  colleges  and  universities  cannot 
systematically  assess  the  performance  of  JPME  research  institutions  and 
whether  they  are  furthering  JPME. 

In  addition  to  the  Joint  Staff’s  accreditation  process,  oversight  of  JPME 
academic  quality  is  performed  by  external  accrediting  bodies. 
Accreditation  is  a  means  of  self-regulation  and  peer  review  to  ensure 
agreed  upon  standards  are  met.  The  regional  accreditation  process  is 
intended  to  examine  academic  institutions  as  a  whole.  While  the 
accreditation  process  may  review  the  extent  to  which  research  is 


"°GAO/AIMD-00-21.3.1. 

^'While  not  tied  to  the  research  institutions,  the  process  provides  an  accreditation  standard 
for  research  related  to  faculty’s  professional  development. 
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conducted  at  a  JPME  college  or  university  among  a  number  of  other 
activities,  these  evaluations  are  not  intended  to  assess  the  performance 
of  JPME  research  institutions  in  meeting  JPME  and  other  departmental 
goals  and  objectives. 

JPME  colleges  and  universities,  as  Master’s  Degree-granting  institutions, 
are  accredited  by  the  following  four  regional  accreditation  bodies: 

•  Middle  States  Commission  on  Higher  Education  accredits  NDU  and 
the  Army  War  College; 

•  Higher  Learning  Commission  accredits  the  Army  Command  and 
General  Staff  College; 

•  New  England  Association  of  Schools  and  Colleges  accredits  the 
Naval  War  College;  and 

•  Southern  Association  of  Colleges  and  Schools  Commission  on 
Colleges  accredits  Air  University,  including  its  Staff  and  War  Colleges, 
and  Marine  Corps  University,  including  its  Staff  and  War  Colleges. 

JPME  colleges  and  universities  are  subject  to  the  regional  accreditation 
processes  every  10  years  and  these  processes  are  intended  to 
strengthen  and  sustain  the  quality  and  integrity  of  higher  education.  For 
example,  according  to  the  Middle  States  Commission  on  Higher 
Education,  accreditation  by  the  commission  is  based  on  the  results  of 
institutional  reviews  by  peers  and  colleagues  and  attests  to  the  judgment 
that  the  institution  has  met  certain  criteria,  such  as  that  it  has  a  mission 
appropriate  to  higher  education;  it  is  guided  by  well-defined  and 
appropriate  goals,  including  goals  for  student  learning;  and  it  has 
established  conditions  and  procedures  under  which  its  mission  and  goals 
can  be  realized. 

While  the  accreditation  process  reviews  the  quality  at  JPME  colleges  and 
universities,  it  does  not  specifically  assess  the  performance  of  JPME 
research  institutions.  Our  review  of  reports  prepared  through  the  regional 


^^While  these  accrediting  bodies  are  institutional  accreditors,  responsible  for  reviewing  an 
entire  institution  and  indicating  that  each  of  an  institution’s  parts  is  contributing  to  the 
achievement  of  the  institution’s  objectives,  their  reviews  do  not  focus  in  detail  on  research 
institutions.  In  addition,  at  some  JPME  schools,  the  research  institutions  are  more-loosely 
affiliated  with  the  school  and  are  not  viewed  as  a  central  part  of  the  institution. 


Page  28 


GAO-14-216  JPME  Research  Institutions 


accreditation  process  as  well  as  interviews  with  JPME  officials 
knowledgeable  about  the  accreditation  processes  confirmed  that  their 
reviews  are  generally  focused  on  the  curriculum  of  JPME  academic 
programs  and  not  the  performance  of  JPME  research  institutions. 
Therefore,  the  accrediting  processes  also  do  not  provide  DOD  or  JPME 
colleges  and  universities  with  a  means  for  evaluating  the  performance  of 
JPME  research  institutions  and  whether  they  are  furthering  JPME  and 
other  departmental  goals. 


DOD  Does  Not 
Formally  Coordinate 
Studies  and  Analysis 
Research  Requests 
Performed  by  JPME 
and  Other  DOD- 
Funded  Research 
Institutions,  Although 
Some  Conduct  Work 
in  Similar  Topic  Areas 


DOD  does  not  formally  coordinate  requests  for  studies  and  analysis 
research  conducted  by  JPME  research  institutions  and  other  DOD-funded 
research  organizations,  even  though  many  of  these  organizations  have 
missions  to  conduct  work  in  similar  topic  areas.  Our  analysis  found  that 
multiple  organizations,  including  JPME  research  institutions  and  other 
DOD-funded  research  organizations,  such  as  Federally  Funded  Research 
and  Development  Centers,  have  missions  to  conduct  work  in  similar  topic 
areas.  However,  DOD  relies  on  a  variety  of  separate  processes  to 
manage  research  requests  that  can  be  conducted  at  either  JPME 
research  institutions  or  other  DOD-funded  research  organizations. 
Specifically,  offices  within  the  Office  of  the  Secretary  of  Defense  and  the 
military  departments  have  their  own  separate  internal  processes  to 
request  such  research.  Because  there  is  no  requirement  for  them  to  do 
so,  these  offices  do  not  have  mechanisms  in  place  to  participate  in  one 
another’s  processes,  thereby  limiting  opportunities  to  share  information 
on  DOD-wide  priorities  and  collective  research  efforts,  and  to  identify  any 
areas  of  potentially  similar  research.  Although  there  are  notable 
differences  even  among  the  JPME  research  institutions  and  other  DOD- 
funded  organizations  that  have  missions  to  conduct  work  in  similar  topic 
areas,  we  note  that,  as  we  concluded  in  September  2009,  organizations 
involved  in  similar  missions  should  coordinate  to  avoid  unnecessary 
duplication  of  work. Furthermore,  results-oriented  management 
practices  call  for  establishing  a  means  to  operate  across  organizational 
boundaries  to  enhance  and  sustain  coordination. 


"'See  GAO-09-904SP. 
""GAO-Oe-IS. 
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Some  DOD  Research 
Organizations  Have 
Missions  to  Conduct  Work 
in  Similar  Topic  Areas 


•  11  JPME  research  institutions,  5  Federally  Funded  Research  and 
Development  Centers,  3  regional  centers  and  2  service-affiliated 
research  organizations  have  missions  to  conduct  research  related  to 
DOD  strategy,  policy,  and  doctrine; 

•  5  JPME  research  institutions,  1  Federally  Funded  Research  and 
Development  Center,  and  3  regional  centers  have  missions  that 
include  researching  issues  related  to  civilian-military  issues  and 
irregular  warfare;  and 

•  2  JPME  research  institutions,  5  Federally  Funded  Research  and 
Development  Centers,  and  4  service-affiliated  research  organizations 
have  missions  that  include  researching  technology,  acquisition,  and 
systems  issues. 

Our  analysis  of  mission  statements  also  identified  instances  in  which 
more-limited  numbers  of  DOD  research  organizations  conduct  work  in 
similar  topic  areas.  For  example,  we  found  that 

•  2  JPME  research  institutions  and  1  service-affiliated  research 
organization  have  missions  that  include  researching  issues  related  to 
cyber  issues;  and 

•  2  JPME  research  institutions  and  2  regional  centers  have  missions 
that  include  researching  issues  related  to  Africa. 

Figure  6  summarizes  the  results  of  our  analysis  of  similarities  in  research 
topic  areas  for  the  20  JPME  research  institutions  and  14  other  DOD- 


Although  we  did  not  identify  specific  instances  of  duplication  through  our 
analyses  of  mission  statements  and  the  research  project  titles  of  20 
JPME  and  14  other  DOD-funded  research  institutions,  we  identified 
similarities  in  their  research  topic  areas.  Through  our  analysis  of  the 
mission  statements,  we  identified  multiple  instances  in  which  several 
DOD  research  organizations  conduct  work  in  similar  topic  areas.'*®  For 
example,  we  found  that 


^^The  topic  areas  represent  23  broad  categories  of  research  activities  and  are  based  on 
the  general  topic  areas  in  which  JPME  research  institutions  and  other  DOD-funded 
research  organizations  categorize  their  research. 
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funded  research  organizations,  according  to  23  areas  of  concentration."^® 
A  checkmark  indicates  a  research  institution’s  mission  statement 
identified  that  category  is  a  topic  area  in  which  it  conducts  research. 


^®The  results  do  not  include  the  following  three  areas  of  concentration:  (1 )  Public  Affairs 
and  Communication;  (2)  Other;  and  (3)  Unable  to  Code.  “Public  Affairs  and 
Communication”  is  not  included  because  no  mission  statements  were  coded  into  that  area 
of  concentration.  “Other”  and  “Unable  to  Code”  were  not  included  because  these  are  not 
areas  of  concentration  intended  to  show  similarity.  Rather,  “Other”  is  intended  for  project 
titles  or  mission  statements  that  do  not  fit  into  the  other  areas  of  concentration  and 
“Unable  to  Code”  is  used  for  methodological  purposes  to  categorize  incomplete 
information. 
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Figure  6:  Similarities  in  Research  Topic  Areas  for  34  Research  Organizations,  According  to  Mission  Statements 
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Note:  See  Abbreviations  list  on  page  Hi. 
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The  similarities  among  the  DOD  research  organizations  are  also 
illustrated  in  the  titles  of  the  research  projects  conducted  by  JPME 
research  institutions  and  other  DOD-funded  research  organizations.  By 
categorizing  2,217  research  project  titles  provided  to  us  for  2012  and 
2013  from  both  JPME  research  institutions  and  other  DOD-funded 
research  organizations,  we  found  that  multiple  organizations’  project  titles 
were  grouped  in  related  topic  areas.  For  example, 

•  project  titles  from  13  JPME  research  institutions,  5  Federally  Funded 
Research  and  Development  Centers,  and  2  regional  centers  were 
related  to  the  Middle  East; 

•  project  titles  from  1 1  JPME  research  institutions,  5  Federally  Funded 
Research  and  Development  Centers,  2  regional  centers,  and  1 
service-affiliated  research  organization  concerned  Asia  studies;  and 

•  project  titles  from  10  JPME  research  institutions,  5  Federally  Funded 
Research  and  Development  Centers,  1  regional  center,  and  1  service- 
affiliated  research  organization  concerned  force  structure  and 
operational  issues. 

Cur  analysis  also  identified  limited  instances  of  similarities  of  specific 
research  project  titles  within  topic  areas.  For  example,  we  identified  four 
research  project  titles  that  focused  on  the  “Arab  Spring”  and  two  research 
project  titles  specifically  related  to  China’s  development-assistance 
efforts.  However,  given  our  objective’s  focus  on  research  organizations  as 
opposed  to  projects,  we  did  not  review  the  content  of  individual  research 
projects  and  their  respective  methodologies.  As  a  result,  we  did  not 
assess  the  extent  to  which  individual  research  projects  and  their  findings 
overlapped  or  were  duplicative  with  other  research  projects.  Appendix  III 
provides  more-detailed  results  of  our  analysis  of  research  project  titles 
conducted  by  JPME  research  institutions  and  other  DCD-funded  research 
organizations  in  2012  and  2013. 


There  Are  Notable 
Differences  among  JPME 
and  Other  DOD-funded 
Research  Institutions 


While  there  are  similarities  in  the  research  topic  areas  of  JPME  research 
institutions  and  other  DOD-funded  research  organizations,  DOD  officials 
also  identified  notable  differences  among  these  organizations.  One  such 
difference  is  that  some  JPME  research  institutions  are  required  to  support 
the  PME  mission  at  their  respective  colleges  and  universities,  whereas 
that  is  not  part  of  the  mission  of  other  DOD-funded  research 
organizations.  For  example,  the  mission  statement  of  NDU’s  Institute  for 
National  Strategic  Studies,  which  comprises  its  research  institutions. 
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currently  includes  advancing  the  strategic  thinking  of  NDU  and  the  JPME 
community  through  research.  Also  along  these  lines,  Air  University’s 
JPME  research  institutions  seek  to  use  research  publications  to  enhance 
strategic  thought  within  the  Air  Force  and  in  Air  University  academic 
curricula.  To  carry  out  their  mission  to  support  JPME  academic  programs, 
the  research  institutions  engage  in  efforts  not  required  of  other-DOD 
funded  research  organizations.  For  example,  officials  at  Air  University 
noted  that  the  research  products  developed  by  the  Counterproliferation 
Center  are  used  to  update  Air  Force  PME  curriculum.  As  another 
example,  faculty  from  Naval  War  College  JPME  research  institutions 
teach  elective  courses  in  the  JPME  academic  program.  Conversely,  the 
mission  of  the  Center  for  Naval  Analyses,  a  Federally  Funded  Research 
and  Development  Center,  is  to  provide  independent,  authoritative 
research,  analysis,  and  technical  support  to  the  Navy  and  other  DOD 
organizations,  and  this  research  is  not  tied  to  JPME  academic  programs. 

We  identified  two  additional  factors  that  differentiate  JPME  research 
institutions  among  themselves  and  other  DOD-funded  research 
organizations.  The  first  pertains  to  differences  among  the  JPME  research 
institutions  as  to  which  office  primarily  sponsors  the  work  of  research 
institutions.  For  example,  two  JPME  research  institutions  have  missions 
to  conduct  research  on  China  issues — the  China  Maritime  Studies 
Institute  at  the  Naval  War  College  and  the  Center  for  the  Study  of 
Chinese  Military  Affairs  at  NDU.  However,  the  two  institutions  conduct 
research  on  different  aspects  of  China,  reflecting  the  interests  of  their 
primary  sponsors.  Specifically,  the  China  Maritime  Studies  Institute 
conducts  research  on  Chinese  maritime  issues  primarily  for  the  Navy, 
while  the  Center  for  the  Study  of  Chinese  Military  Affairs  conducts 
broader  research  on  Chinese  strategic-level  issues  for  the  Joint  Staff  and 
Office  of  the  Secretary  of  Defense. 

According  to  officials  from  JPME  colleges  and  universities  and  other 
DOD-funded  research  organizations,  a  second  factor  that  differentiates 
JPME  research  institutions  among  themselves  and  with  other  DOD- 
funded  research  organizations  is  the  level  of  technical  expertise  provided 
by  some  research  organizations.  Specifically,  officials  explained  that 
Federally  Funded  Research  and  Development  Centers  can  produce 
research  with  a  more  scientific  and  technical  focus  than  that  of  JPME 
institutions.  For  example,  while  both  the  Institute  for  Defense  Analyses,  a 
Federally  Funded  Research  and  Development  Center,  and  NDU’s  Center 
for  Technology  and  National  Security  Policy  have  missions  related  to 
researching  technology,  the  Institute  for  Defense  Analyses  conducts  tests 
and  evaluations  of  technologies,  requiring  staff  to  have  specialized 
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scientific  and  technical  skills,  while  the  Center  for  Technology  and 
National  Security  Policy’s  research  discusses  the  effect  of  technology  on 
defense  policy. 


Multiple  Offices  Use 
Fragmented  Processes  to 
Request  Studies  and 
Analysis  Research 


Although  multiple  organizations,  including  JPME  research  institutions  and 
other  DOD-funded  research  organizations,  have  missions  to  conduct 
work  in  similar  topic  areas,  offices  throughout  DOD  use  separate 
processes  to  request  studies  and  analysis  research.  This  fragmentation 
across  DOD  occurs  in  the  absence  both  of  a  DOD  requirement  to 
coordinate  studies  and  analysis  research  requirements  among  the  military 
departments  and  of  other  DOD  offices  and  mechanisms  to  facilitate  such 
coordination.  In  September  2009,  we  concluded  that  offices  involved  in 
similar  missions  should  coordinate  and  share  relevant  information  to 
avoid  unnecessary  duplication  of  work. Furthermore,  results-oriented 
management  practices  call  for  establishing  a  means  to  operate  across 
organizational  boundaries  to  enhance  and  sustain  coordination.'*® 


We  identified  several  separate  processes  used  by  JPME  research 
institutions  or  DOD  offices  to  manage  requests  for  studies  and  analysis 
research,  but  DOD  has  not  established  formal  mechanisms  to  coordinate 
requests.  JPME  research  institutions,  for  example,  individually  manage 
their  own  research  activities.  According  to  Joint  Staff  officials,  JPME 
research  activities  are  not  typically  coordinated  with  other  departmental 
offices  that  request  studies  and  analysis  research.  At  JPME  research 
institutions,  researchers  have  the  discretion  to  determine  whether 
research  has  been  or  is  being  conducted  on  a  given  topic.  For  example, 
JPME  research  institution  officials  told  us  that  while  it  is  not  a 
requirement,  they  may  contact  other  subject-matter  experts  to  determine 
whether  similar  work  is  being  conducted  at  another  JPME  research 
institution.  Officials  also  said  researchers  may  conduct  a  literature  review 
to  understand  the  existing  research  on  a  topic  as  part  of  the  research 
process,  or  they  may  review  completed  research  projects  that  are 
contained  in  the  Defense  Technical  Information  Center  database  to  see 


"'See  GAO-09-904SP. 
"®GAO-06-15. 
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whether  DOD  has  funded  past  studies. However,  that  database  does 
not  contain  information  on  ongoing  research  efforts,  and  no  other  formal 
mechanism  for  sharing  information  on  ongoing  studies  and  analysis 
research  activities  within  DOD  was  identified. 

Within  the  Office  of  the  Secretary  of  Defense,  multiple  offices  generate 
requests  annually  for  studies  and  analysis  research,  but  these  research 
requests  are  determined  based  on  individual  offices’  research 
requirements  and  are  not  formally  coordinated  with  other  departmental 
offices.  Office  of  the  Secretary  of  Defense  research  requests  may  be 
fulfilled  by  contracting  with  other  DOD-funded  research  organizations  or 
JPME  research  institutions  to  conduct  the  research.  For  example, 
research  requests  for  the  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology  and  Logistics  are  managed  at  the  Office  of  the 
Secretary  of  Defense,  Studies  and  Federally  Funded  Research  and 
Development  Center  Management  office.  In  doing  so,  officials  with  this 
office  explained  that  they  do  not  formally  coordinate  with  other  DOD 
offices  to  determine  whether  similar  research  requests  are  being  funded 
by  other  departmental  offices.  Separately,  the  Office  of  the  Under 
Secretary  of  Defense  for  Policy  uses  a  different  process  to  manage 
requests  for  studies  and  analysis  research.  Specifically,  the  Office  of  the 
Deputy  Under  Secretary  of  Defense  for  Strategy,  Plans,  and  Forces 
reviews  requests  from  within  the  Office  of  the  Under  Secretary  of  Defense 
for  Policy  on  an  annual  basis.  These  research  requests  are  not  formally 
coordinated  with  other  Office  of  the  Secretary  of  Defense  offices  or  other 
departmental  offices,  such  as  service-level  studies  and  analysis  offices,  to 
determine  whether  similar  work  is  being  conducted  or  funded  elsewhere. 

Similarly,  the  military  departments  have  their  own  respective  internal 
processes  for  requesting  studies  and  analysis  research,  but  absent  a 
DOD  requirement  to  do  so,  these  processes  are  not  used  to  formally 
coordinate  research  requests  among  the  military  departments  or  with 
other  DOD  offices.  In  general,  these  processes  are  used  as  a  mechanism 
to  coordinate  requests  for  studies  and  analysis  research  within  each  of 
the  military  departments.  For  example: 


^^The  Defense  Technical  Information  Center  manages  an  online  database  that  makes  past 
DOD-funded  research  studies  available  to  the  research  community  to  enable  future 
researchers  to  understand  the  purpose,  scope,  approach,  results  or  outcomes,  and 
conclusions  or  recommendations  of  prior  work  before  undertaking  new  studies. 


Page  36 


GAO-14-216  JPME  Research  Institutions 


•  According  to  a  senior  Air  Force  official,  the  Air  Staff’s  Analyses, 
Assessments  and  Lessons  Learned  directorate  is  responsible  for 
collecting  annual  research  requests  from  across  the  Air  Force  and  for 
ensuring  that  the  contracted  studies  are  not  duplicative.  A  senior 
official  within  this  office  told  us  that  the  Air  Force  has  experienced 
challenges  with  regard  to  its  oversight  over  the  number  of  studies  it 
has  funded.  In  response,  the  Air  Force  has  developed  a  policy  to  track 
all  Air  Force  funded  studies  in  an  internal  database.  However, 
according  to  this  official,  the  Analyses,  Assessments  and  Lessons 
Learned  directorate  generally  does  not  formally  coordinate  with  offices 
outside  the  Air  Force  on  annual  research  requests. 

•  The  Army  Study  Program  Management  Office  within  Headquarters, 
Department  of  the  Army,  issues  an  annual  call  for  research  requests 
from  Army  commands,  and  in  turn  funds  the  research  requests 
according  to  Army  priorities.  A  senior  Army  official  in  this  office  said 
that  its  process  is  focused  on  reviewing  Army-specific  research 
requests  and  does  not  include  other  DOD  offices  that  request  or 
conduct  studies  and  analysis  research. 

•  The  Navy’s  annual  research  requests  are  administered  through  the 
Office  of  the  Chief  of  Naval  Operations,  which  compiles  and  prioritizes 
research  needs  identified  from  across  the  Navy.  The  Navy’s  studies 
and  analysis  program  guidance  says  that  the  Navy  should  coordinate 
analytic  efforts  with  the  Marine  Corps,  but  according  to  an  official  in 
this  office,  the  Navy  generally  does  not  coordinate  with  other  DOD- 
funded  research  organizations  or  JPME  research  institutions  with 
regard  to  these  annual  research  requests. 

DOD  officials  within  the  studies  and  analysis  research  community 
observed  that  there  are  both  costs  and  benefits  to  the  department’s 
decentralized  approach  to  requesting  studies  and  analysis  research.  One 
official  told  us  that  limited  coordination  among  the  multiple  offices  that 
request  studies  and  analysis  research  may  put  DOD  at  risk  for  funding 
overlapping  research  activities.  Furthermore,  a  senior  Air  Force  official  in 
the  Air  Staff’s  Analyses,  Assessments  and  Lessons  Learned  directorate 
stated  that  the  current  approach  makes  it  difficult  for  DOD  to  have  a 
complete  picture  of  how  much  money  is  being  spent  on  studies  and 
analysis  research.  While  DOD  officials  identified  costs  to  the  current 
approach  for  coordinating  studies  and  analysis  research  requests, 
officials  also  acknowledged  that  DOD’s  decentralized  approach  may 
result  in  several  benefits.  For  example,  a  senior  Air  Force  official  in  the  Air 
Staff’s  Analyses,  Assessments  and  Lessons  Learned  directorate  stated 
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that  decentralization  generates  creativity  and  diversity  of  thought  in 
DOD’s  studies  and  analysis  community,  which  can  prove  useful  in 
informing  DOD  decision  makers.  Furthermore,  a  senior  Navy  official  in  the 
Office  of  the  Chief  of  Naval  Operations  stated  that  the  current  approach 
allows  each  office  to  be  concerned  with  its  own  area  of  functional 
expertise,  which  varies  widely  across  the  services  and  DOD.  For 
example,  the  Office  of  the  Chief  of  Naval  Operations  is  responsible  for 
funding  studies  related  to  naval  functional  areas  such  as  designing, 
building,  and  maintaining  ships,  which  is  unique  when  compared  to  other 
service-level  studies  and  analysis  offices  such  as  the  Air  Force  Analyses, 
Assessments  and  Lessons  Learned  office  or  the  Army  Study  Program 
Management  Office. 

In  contrast  to  how  it  manages  requests  for  studies  and  analysis  research, 
DOD  has  established  mechanisms  to  coordinate  science  and  technology- 
specific  research  efforts  across  multiple  departmental  offices  engaged  in 
similar  missions.  Specifically,  the  science  and  technology  research 
community  has  governing  bodies,  such  as  executive  committees  to 
facilitate  such  coordination.  These  committees  are  intended  to  better 
manage  DOD’s  science  and  technology  research  by  bringing  together  the 
multiple  departmental  offices  that  sponsor  such  research  for  the  purpose 
of  sharing  information.  A  senior  official  responsible  for  coordinating 
science  and  technology  research  efforts  explained  that  the  executive 
committees  do  not  require  additional  resources.  Rather,  they  are  intended 
to  share  offices’  existing  annual  research  plans  and  provide  opportunities 
to  leverage  resources  in  a  fiscally  constrained  environment.  Some  DOD 
officials  we  spoke  with  who  are  responsible  for  managing  studies  and 
analysis  research  requests  also  said  that  a  mechanism  that  provided 
greater  information  on  what  studies  and  analysis  research  other 
departmental  offices  were  sponsoring  would  improve  their  ability  to 
identify  potential  overlap  in  research  requests.  Without  a  mechanism  for 
coordinating  research  requests  and  sharing  information  on  studies  and 
analysis  research  activities  among  multiple  offices,  DOD  cannot  ensure 
that  it  minimizes  potentially  unnecessary  overlap  in  research  activities. 
Furthermore,  making  information  on  department-wide  annual  research 
requests  available  to  JPME  research  institutions  would  provide  the 
institutions  an  opportunity  to  further  understand  research  needs  and  align 
some  of  the  institutions’  research  with  strategic  priorities  identified  by 
DOD  leadership. 
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Conclusions 


Given  the  ongoing  and  unique  role  of  JPME  research  institutions  in  the 
development  of  DOD’s  future  leaders,  DOD’s  oversight  of  these 
institutions  is  important  for  helping  the  department  to  make  the  best  use 
of  the  resources  it  devotes  to  the  colleges  and  universities  that  provide 
PME  and  JPME  and  for  decreasing  fragmentation  of  research  requests 
and  the  risk  of  potential  overlap  in  research  activities.  As  fiscal  pressures 
facing  DOD  continue  to  mount,  so  too  does  the  need  for  the  department 
to  prioritize  resources  for  JPME  research  institutions  to  most-effectively 
meet  the  JPME  mission.  Considering  the  overall  growth  of  JPME 
research  institutions  that  occurred  between  fiscal  year  2007  and  fiscal 
year  201 1  in  number,  funding,  and  size,  it  is  paramount  that  DOD 
maintains  oversight  of  these  institutions.  Best  practices  state  that 
achieving  results  in  government  requires  a  comprehensive  oversight 
framework  that  includes  measurable  goals  and  objectives,  and  metrics  for 
assessing  progress.  Yet,  with  limited  exceptions,  the  research  institutions 
lack  such  goals,  objectives,  and  associated  metrics.  Additionally,  while 
DOD  has  some  oversight  mechanisms  in  place  for  JPME,  DOD  does  not 
have  clear  guidance  establishing  the  role  of  JPME  research  institutions  in 
furthering  PME  or  other  departmental  goals  that  would  provide  a  basis  for 
evaluating  their  performance  and  helping  ensure  that  intended  results  are 
achieved.  Further,  no  entity  within  DOD  is  assigned  responsibility  for 
overseeing  the  performance  of  JPME  research  institutions.  Consequently, 
DOD  cannot  ensure  the  effectiveness  of  JPME  research  institutions  and 
lacks  a  sound  basis  for  making  resource  determinations.  Furthermore, 
some  JPME  research  institutions  and  other  DOD  funded  research 
organizations  have  missions  to  conduct  research  in  similar  topic  areas, 
but  DOD  uses  a  variety  of  separate  processes  for  requesting  studies  and 
analysis  research.  Results-oriented  management  practices  call  for 
establishing  a  means  to  operate  across  organizational  boundaries  to 
enhance  and  sustain  coordination.  DOD,  however,  does  not  have  a 
mechanism  in  place  to  coordinate  studies  and  analysis  research  requests 
and  minimize  fragmentation.  While  DOD  officials  believe  that  their  current 
decentralized  approach  to  requesting  studies  and  analysis  research  has 
its  benefits,  they  also  recognize  that  it  has  its  costs.  It  is  not  clear  that  the 
benefits  of  DOD’s  current  approach  outweigh  the  risks  of  fragmentation 
and  potential  duplication,  particularly  in  a  budget-constrained 
environment.  DOD’s  science  and  technology  research  community 
provides  one  mechanism  for  a  coordination  mechanism,  but  by  no  means 
is  it  the  only  mechanism  that  could  meet  the  needs  of  the  studies  and 
analysis  research  community  as  it  seeks  to  support  department-wide 
priorities.  Without  a  mechanism  to  facilitate  coordination  and  reduce 
fragmentation  among  offices  requesting  studies  and  analysis  research, 
DOD  cannot  ensure  that  it  minimizes  potential  overlap  in  research 
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activities  and  that  its  resources  are  used  efficiently  in  support  of 
department-wide  priorities  at  its  JPME  research  institutions  and  other 
research  organizations. 


Recommendations 
Executive  Action 


•  define  the  role  of  JPME  research  institutions  to  provide  a  basis  for 
evaluating  their  performance, 

•  assign  responsibilities  for  conducting  performance  reviews  of  JPME 
research  institutions,  and 

•  establish  a  framework  that  includes  measurable  goals  and  objectives 
linked  with  metrics  to  assess  the  performance  of  JPME  research 
institutions. 

To  improve  the  coordination  of  requests  for  studies  and  analysis  research 
within  the  department  and  to  reduce  the  risk  of  potential  overlap  in 
research  activities,  we  recommend  that  the  Secretary  of  Defense 
establish  and  implement  a  departmental  mechanism  that  requires 
leadership  from  the  military  services  and  departmental  offices  responsible 
for  managing  requests  for  studies  and  analysis  research  to  coordinate 
their  annual  research  requests  and  ongoing  research  efforts. 


To  enhance  the  performance  of  JPME  research  institutions,  we  are 
recommending  that  the  Secretary  of  Defense  direct  the  Chairman  of  the 
Joint  Chiefs  of  Staff  and  the  Secretaries  of  the  military  departments  for 
their  respective  PME  and  JPME  colleges  and  universities  to  take  the 
following  three  actions: 


Agency  Comments 
and  Our  Evaluation 


In  written  comments  on  a  draft  of  this  report,  DOD  concurred  with  our 
recommendations.  The  full  text  of  DOD’s  written  comments  is  reprinted  in 
appendix  IV. 

In  concurring  with  our  first  recommendation,  DOD  noted  that  some  work 
is  already  in  progress  to  clarify  organizational  goals  and  establish  metrics 
of  success  at  each  of  the  research  institutions.  DOD  stated  that,  for 
example,  the  Joint  Staff  has  been  collaborating  with  NDU  to  refine  its 
research  enterprise.  DOD  noted  that  our  recommendation  is  reflected  in 
NDU’s  Strategic  Plan  for  Research  2014-2019  and  a  revision  of  the 
Chairman’s  policy  document  for  the  university.  According  to  the 
department,  our  recommendation  should  be  fully  implemented  when  the 
next  academic  year  begins  in  the  fall  of  2014.  We  agree  that  these  are 
positive  steps  towards  establishing  a  comprehensive  framework  at  NDU 
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to  systematically  assess  the  performance  of  its  JPME  research 
institutions  in  meeting  PME  and  other  departmental  goals  and  objectives. 
Notwithstanding  this  effort,  as  noted  in  our  report  there  remains  no  DOD- 
wide  guidance  that  addresses  the  intended  role  of  research  institutions  in 
supporting  PME,  including  JPME,  or  other  departmental  goals  or  assigns 
responsibilities  for  conducting  performance  reviews  of  them.  This  leaves 
the  department  without  a  sound  basis  to  assess  NDU’s  and  the  other 
research  institutions’  stated  missions  and  actual  performance  against 
planned  or  expected  results.  Clearly  establishing  linkages  between 
significant  activities,  their  intended  role  in  meeting  agency-wide  goals  and 
objectives,  and  assigning  oversight  responsibilities  underpins  DOD’s 
ability  to  conduct  such  reviews. 

In  its  concurrence  with  our  second  recommendation,  DOD  stated  that  to 
improve  coordination  of  research  requests,  it  plans  to  establish  a  Studies 
and  Analysis  Executive  Committee  by  the  end  of  fiscal  year  2014  with 
regional  and  topical  “communities  of  interest.”  DOD  noted  that  the 
committee  will  be  a  combined  effort  organized  through  the  Office  of  the 
Under  Secretary  of  Defense  for  Policy  and  the  Assistant  Secretary  of 
Defense  for  Research  and  Engineering,  with  other  representation  from 
the  JPME  and  PME  community,  as  appropriate. 

DOD  also  provided  technical  comments  on  a  draft  of  our  report,  which 
we  have  incorporated  into  the  report,  as  appropriate. 


We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees.  We  are  also  sending  copies  to  the  Secretary  of  Defense,  the 
Chairman  of  the  Joint  Chiefs  of  Staff,  the  Secretary  of  the  Air  Force,  the 
Secretary  of  the  Army,  the  Secretary  of  the  Navy,  and  the  Commandant 
of  the  Marine  Corps.  In  addition,  the  report  will  also  be  available  at  no 
charge  on  the  GAO  website  at  http://www.gao.gov. 

If  you  or  your  staff  have  questions  about  this  report,  please  contact  me  at 
(202)  512-5741  or  ayersj@gao.gov.  Contact  points  for  our  Offices  of 
Congressional  Relations  and  Public  Affairs  may  be  found  on  the  last  page 
of  this  report.  Key  contributors  to  this  report  are  listed  in  appendix  V. 


Johana  Ayers 
Acting  Director 

Defense  Capabilities  and  Management 
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The  Honorable  Carl  Levin 
Chairman 

The  Honorable  James  Inhofe 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Richard  Durbin 
Chairman 

The  Honorable  Thad  Cochran 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Howard  P.  McKeon 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Rodney  Frelinghuysen 
Chairman 

The  Honorable  Pete  Visclosky 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 


Page  42 


GAO-14-216  JPME  Research  Institutions 


Appendix  I:  Objectives,  Scope,  and 
Methodology 


The  National  Defense  Authorization  Act  for  Fiscal  Year  2013  mandated 
that  we  review  the  work  performed  by  joint  professional  military  education 
(JPME)  research  institutions  in  support  of  professional  military  education 
and  the  Department  of  Defense’s  (DOD)  broader  mission.''  In  this  report, 
we  (1)  describe  how  JPME  research  institutions  have  changed  in  number, 
funding,  and  size,  and  the  factors  that  contributed  to  any  changes;  (2) 
evaluate  the  extent  to  which  DOD  is  assessing  the  performance  of  JPME 
research  institutions  in  meeting  professional  military  education  and  other 
departmental  goals  and  objectives;  and  (3)  evaluate  the  extent  to  which 
DOD  coordinates  research  requests  for  JPME  research  institutions  and 
other  DOD-funded  research  organizations. 

To  address  these  objectives,  we  included  in  the  scope  of  our  review  the 
intermediate-  and  senior-level  colleges  and  universities  that  provide 
JPME  certification — that  is,  the  National  Defense  University,  Army  War 
College,  Army  Command  and  General  Staff  College,  Air  University,  Naval 
War  College,  and  Marine  Corps  University.^  Research  conducted  at 
JPME  colleges  and  universities  can  be  accomplished  by  research 
institutions  associated  with  the  colleges  and  universities  as  well  as  by 
students  and  professors  as  part  of  academic  coursework.  For  the 
purposes  of  this  report,  we  focused  on  research  institutions  that  conduct 
research  as  their  primary  mission  and  have  dedicated  personnel  to  do  so. 
We  excluded  certain  centers  and  institutes  associated  with  JPME 
colleges  and  universities  because  we  determined  they  did  not  have  a 
mission  to  primarily  conduct  research  or  did  not  have  dedicated 
personnel.^  We  also  excluded  from  our  analysis  research  conducted  by 
students  as  part  of  their  academic  coursework  and  by  professors  not 
affiliated  with  the  research  institutions.  To  identify  the  JPME  research 
institutions  included  within  the  scope  of  our  review,  we  examined 
documentation  and  gathered  testimonial  evidence  related  to  the  missions, 
activities,  and  organization  of  the  institutions  that  conduct  research  at  the 


^Pub.  L  No.  112-239,  §  547(b)  (2013). 

^For  the  purposes  of  this  report,  we  refer  to  service  and  joint  colleges  and  universities  that 
are  accredited  by  the  Joint  Staff  to  provide  JPME  certification  as  JPME  colleges  and 
universities. 

^For  example,  although  the  Army  War  College’s  Army  Physical  Fitness  Research  Institute 
conducted  research  and  had  dedicated  personnel,  we  concluded  that  its  resources  were 
primarily  focused  on  educating  and  promoting  the  mental  and  physical  readiness  of 
thousands  of  Army  leaders  and  spouses  each  year.  As  a  result,  we  concluded  that  it  did 
not  meet  both  of  our  criteria  and  is  not  included  within  our  scope. 
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JPME  colleges  and  universities.  Based  on  this  evidence,  we  determined 
that  20  institutions  were  conducting  research  as  their  primary  mission  and 
had  dedicated  personnel  assigned  to  them  and  therefore  were  included  in 
the  scope  of  our  review. 

For  our  first  objective  of  determining  the  extent  to  which  JPME  research 
institutions  have  changed  in  number,  funding,  and  size,  and  the  factors 
contributing  to  any  changes,  we  obtained  questionnaire  responses  and 
other  documentation  on  the  number  of  research  institutions  that  existed  at 
JPME  colleges  and  universities  from  fiscal  years  2000  through  2013  and 
collected  and  analyzed  available  funding  and  staffing  data  for  these 
years'^  for  the  JPME  research  institutions.  We  assessed  the  reliability  of 
the  funding  and  staffing  data  collected  by  analyzing  questionnaire 
responses  from  JPME  colleges  and  universities,  which  included 
information  on  their  data-system  management,  data  quality-assurance 
processes,  and  potential  sources  of  errors  and  mitigations  of  those  errors. 
Based  on  our  review  of  the  data  provided  and  our  review  of  the 
questionnaire  responses,  we  concluded  that  the  systems  used  to  provide 
the  data,  and  thus  the  data  they  provide,  are  sufficiently  reliable  for  our 
audit  purposes.  However,  based  on  this  evidence,  we  determined  that  we 
were  unable  to  report  consistent  data  on  JPME  research  institution 
funding  and  staffing  levels  prior  to  fiscal  year  2007,  and  therefore  we  are 
providing  trend  data  on  JPME  research  institutions  from  fiscal  years  2007 
through  2013.^  Furthermore,  although  we  have  identified  a  number  of 
factors  that  could  affect  data  quality,  we  concluded  that  these  were  the 
best  available  data  on  JPME  research  institutions.  We  also  concluded 
that  the  data  would  not  lead  to  an  incorrect  or  unintentional  message 
since  it  is  corroborated  through  interviews  with  cognizant  officials  at  the 
National  Defense  University,  Army  War  College,  Army  Command  and 
General  Staff  College,  Air  University,  Naval  War  College,  and  Marine 
Corps  University.  We  also  discussed  the  reasons  for  any  trends  in  these 
budget  and  staffing  data  with  knowledgeable  officials  in  DOD  and  at  the 
JPME  colleges  and  universities.  Specifically,  we  conducted  interviews 
with  officials  from  the  Joint  Staff;  Office  of  the  Under  Secretary  of  Defense 


^staffing  data  provided  by  JPME  colleges  and  universities  includes  military  and  civilian 
personnel,  noncontractor  researchers,  contractor  researchers  as  well  as  noncontractor 
administrative  staff  positions. 

^Officials  at  JPME  colleges  and  universities  stated  because  federal  records-retention 
policies  generally  require  agencies  to  maintain  data  for  6  years  and  3  months,  they  were 
unable  to  provide  reliable  data  in  many  cases  prior  to  fiscal  year  2007. 
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(Comptroller);  National  Defense  University;  Army  War  College;  Army 
Command  and  General  Staff  College;  Air  University;  Naval  War  College; 
and  Marine  Corps  University. 

For  our  second  objective  of  determining  the  extent  to  which  DOD  is 
assessing  the  performance  of  JPME  research  institutions  in  meeting 
professional  military  education  and  other  departmental  goals  and 
objectives,  we  obtained  and  reviewed  documentation  from  the  Joint  Staff 
and  the  JPME  colleges  and  universities  that  identify  any  goals,  objectives, 
or  performance  measures  for  JPME  research  institutions.  Specifically,  we 
reviewed  current  strategic  plans,  mission  statements,  and  other 
documentation  describing  activities  of  the  JPME  research  institutions.  We 
also  reviewed  documentation  describing  oversight  mechanisms  that 
monitor  the  academic  quality  of  JPME  colleges  and  universities. 
Specifically,  we  reviewed  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction 
(CJCSI)  1 800.01  D,  Officer  Professional  Military  Education  Policy,  which  is 
the  DOD  policy  that  governs  the  Joint  Staff’s  Accreditation  of  Joint 
Education  process.®  We  also  reviewed  reports  prepared  by  external 
regional  accrediting  bodies.^  To  further  our  understanding  of  any 
processes  used  to  assess  the  performance  of  JPME  research  institutions, 
we  used  a  standard  set  of  questions  to  interview  officials  with  the  Joint 
Staff;  Army,  Navy,  Marine  Corps,  and  Air  Force;  National  Defense 
University;  Army  War  College;  Army  Command  and  General  Staff 
College;  Air  University;  Naval  War  College;  and  Marine  Corps  University. 
We  then  reviewed  the  results  of  the  interviews  and  related  documents  to 
develop  summary  findings.  In  reviewing  this  documentation  and 
testimonial  evidence,  we  referred  to  our  prior  work  on  best  practices  that 
identifies  elements  that  constitute  a  comprehensive  oversight  framework. 


^Department  of  Defense,  Chairman  of  the  Joint  Chiefs  of  Staff,  Officer  Professional  Military 
Education  Policy,  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  (CJCSI)  1 800.01  D 
(Washington,  D.C.:  Dec.  15,2011). 

^The  JPME  colleges  and  universities,  as  Master’s  Degree-granting  institutions,  are 
accredited  by  four  regional  accreditation  bodies:  the  Middle  States  Commission  on  Higher 
Education  (which  accredits  the  National  Defense  University  and  Army  War  College); 

Higher  Learning  Commission  (which  accredits  the  Army  Command  and  General  Staff 
College);  New  England  Association  of  Schools  and  Colleges  (which  accredits  the  Naval 
War  College);  and  Southern  Association  of  Colleges  and  Schools  Commission  on 
Colleges  (which  accredits  Air  University  and  Marine  Corps  University).  The  accreditation 
process  is  intended  to  strengthen  and  sustain  the  quality  and  integrity  of  higher  education, 
making  it  worthy  of  public  confidence  and  minimizing  the  scope  of  external  control. 


Page  45 


GAO-14-216  JPME  Research  Institutions 


Appendix  I:  Objectives,  Scope,  and 
Methodology 


Specifically,  best  practices®  state  that  such  a  framework  should  include 
measurable  goals  and  objectives  linked  with  metrics  for  assessing 
progress,  which  is  consistent  with  the  framework  identified  in  the 
Government  Performance  and  Results  Act  (GPRA),  as  amended  by  the 
GPRA  Modernization  Act  of  2010.®  We  also  reviewed  this  evidence  in 
light  of  key  internal  control  standards,  which  state  that  federal  agencies 
should  conduct  reviews  by  management  at  the  functional  or  activity  level 
and  compare  actual  performance  to  planned  or  expected  results.''® 

For  our  third  objective  of  determining  the  extent  to  which  DOD 
coordinates  research  requests  for  JPME  research  institutions  and  other 
DOD-funded  research  organizations,  we  included  the  20  JPME  research 
institutions  discussed  above  and  14  other  DOD-funded  research 
organizations.  To  identify  the  other  DOD-funded  research  institutions  to 
include  in  the  scope  of  our  review,  we  gathered  documentation  from  the 
Assistant  Secretary  of  Defense  for  Research  and  Engineering  and 
canvassed  knowledgeable  DOD  officials  in  offices  responsible  for 
requesting  research,  such  as  the  military  departments,  science  and 
technology  executive  agents,  studies  and  analysis  research  program 
managers,  and  the  Office  of  the  Secretary  of  Defense,  including  the 
offices  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and 
Logistics  and  the  Under  Secretary  of  Defense  for  Policy.  Based  on  this 
work,  we  determined  that  we  would  include  the  following  14  other  DOD- 
funded  research  organizations  for  the  purposes  of  our  review: 

•  DOD’s  Regional  Centers  for  Security  Studies:'’'' 

•  George  C.  Marshall  European  Center  for  Security  Studies, 


*See,  for  example,  GAO,  Building  Partner  Capacity:  Actions  Needed  to  Strengthen  DOD 
Efforts  to  Assess  the  Performance  of  the  Regional  Centers  for  Security  Studies, 

GAO-1 3-606  (Washington,  D.C.:  June  28,  2013)  and  State  Partnership  Program: 
Improved  Oversight,  Guidance,  and  Training  Needed  for  National  Guard’s  Efforts  with 
Foreign  Partners,  GAO-12-548  (Washington,  D.C.:  May  15,  2012). 

®Pub.  L  No.  103-62  (1993);  Pub.  L  No.  111-352  (2011). 

“GAO,  Standards  for  Internal  Control  in  the  Federal  Government,  GAO/AIMD-00-21.3.1. 
(Washington,  D.C.:  November  1999). 

“We  did  not  include  the  Asia-Pacific  Center  for  Security  Studies  because  it  did  not  have 
any  research  projects  in  2012  or  2013  that  could  be  used  in  our  analysis  of  research 
project  titles  as  part  of  our  effort,  as  discussed  below,  to  identify  similarities  and 
differences  between  research  organizations. 
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•  the  Near  East  South  Asia  Center  for  Strategic  Studies, 

•  the  William  J.  Perry  Center  for  Hemispheric  Defense  Studies,  and 

•  the  Africa  Center  for  Strategic  Studies; 

•  service-affiliated  research  organizations: 

•  the  Naval  Postgraduate  School  Modeling,  Virtual  Environments  and 
Simulation  Institute, 

•  the  Naval  Postgraduate  School’s  Center  for  Interdisciplinary  Remotely 
Piloted  Aircraft  Studies, 

•  the  Naval  Postgraduate  School’s  Cebrowski  Institute  for  Information 
and  Innovation, 

•  the  Army’s  Center  for  Army  Analysis,  and 

•  U.S.  Army  Training  and  Doctrine  Command  Analysis  Center; 

•  Federally  Funded  Research  and  Development  Centers  that  were 
identified  as  studies  and  analysis  research  centers: 

•  Center  for  Naval  Analyses, 

.  RAND  Arroyo, 

•  RAND  Project  Air  Force, 

•  RAND  National  Defense  Research  Institute,  and 

•  the  Institute  for  Defense  Analyses. 

To  further  our  understanding  on  the  processes  used  to  request  studies 
and  analysis  research  from  JPME  institutions  and  other  DOD-funded 
research  organizations,  we  reviewed  documentation  from  and  interviewed 
officials  at  the  military  services.  Office  of  the  Secretary  of  Defense,  JPME 
research  institutions  and  other  DOD-funded  research  organizations.  We 
also  reviewed  documentation  and  interviewed  knowledgeable  agency 
officials  about  DOD’s  approach  to  coordinate  research  requests  among 
DOD  organizations.  Specifically,  we  used  a  standard  set  of  questions  to 
interview  officials  with  the  Office  of  the  Secretary  of  Defense  and  the 
military  services  about  the  processes  used  to  share  information  with  other 
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offices  that  also  request  studies  and  analysis  research.  We  then  reviewed 
the  results  of  the  interviews  and  related  documents  to  develop  summary 
findings.  We  reviewed  the  documentary  and  testimonial  evidence  in  light 
of  key  practices  for  enhancing  and  sustaining  coordination  as  described 
in  best  practices.  Specifically,  best  practices  state  that  organizations 
involved  in  similar  missions  should  coordinate  and  share  information  to 
avoid  unnecessary  duplication  of  work. Further,  we  assessed  whether 
there  were  any  similarities  or  dissimilarities  among  the  missions  of  the 
JPME  research  institutions  and  other  DOD-funded  research 
organizations.  We  did  this  in  four  ways: 

•  First,  we  assessed  whether  the  JPME  and  DOD-funded  research 
institutions  were  conducting  science  and  technology-related  research 
or  studies  and  analysis-related  research,  using  missions  statements 
and  other  documentation  provided  by  DOD  to  make  this 
determination.  We  did  not  conduct  further  analysis  on  other  DOD- 
funded  organizations  that  conduct  science  and  technology  research 
as  that  type  of  research  represented  a  notable  difference  from  the 
JPME  research  institutions  that  primarily  conduct  studies  and  analysis 
research. 

•  Second,  for  all  the  JPME  and  DOD-funded  research  institutions 
conducting  studies  and  analysis  research,  we  reviewed  mission 
statements  and  other  mission-related  information  provided  by  DOD, 
and  categorized  the  organization’s  mission  as  falling  primarily  into  1  or 
more  of  23  areas  of  concentration — ^for  example,  Asia  studies  or 
leadership  and  ethics  studies.  To  create  the  23  areas  of  concentration 
that  were  used  to  categorize  mission  statements  and  research  project 
titles,  we  reviewed  documentation  from  the  JPME  research  institutions 
that  identified  general  topic  areas  in  which  the  institutions  conducted 
research.  We  also  reviewed  documentation  from  Federally  Funded 
Research  and  Development  Centers  that  identified  the  core  topic 
areas  within  which  the  Federally  Funded  Research  and  Development 
Centers  were  authorized  to  conduct  work.  The  areas  of  concentration 
are  identified  as  follows:  Africa;  Asia;  Europe;  Middle  East;  Western 
Hemisphere;  civilian-military  issues  and  irregular  warfare;  cyber; 
energy  and  environment;  force  structure  and  operational  issues; 


^^GAO,  Interagency  Collaboration:  Key  Issues  for  Congressional  Oversight  of  National 
Security  Strategies,  Organizations,  Workforce,  and  Information  Sharing,  GAO-09-904SP 
(Washington,  D.C.:  Sept.  25,  2009). 
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historical;  intelligence;  leadership  and  ethics;  legal;  logistics;  nuclear 
and  weapons  of  mass  destruction;  other;  personnel  and  training; 
public  affairs  and  communication;  resource  management;  strategy, 
policy,  and  doctrine;  technology,  acquisition,  and  systems;  war 
gaming;  and  unable  to  code.  We  determined  that  these  areas  of 
concentration  we  selected  were  appropriate  for  comparing  JPME 
research  institutions  and  other  DOD-funded  research  institutions 
because  they  explain  the  focus  of  each  organization’s  primary  studies 
and  analysis  efforts.  To  complete  the  content  analysis,  two  GAO 
analysts  independently  reviewed  the  mission  statements  and  other 
mission-related  information  provided  by  DOD  and  coded  them  into 
one  or  more  of  the  23  areas  of  concentration.  When  the  coding  was 
completed,  both  analysts  reviewed  every  code  made  by  the  other 
analyst  and  indicated  whether  they  agreed  or  disagreed  with  the  code. 
The  analysts  then  met  to  discuss  their  coding  determinations  and  to 
reach  agreement  where  there  were  any  discrepancies.  The  results  of 
our  analysis  are  not  generalizable  beyond  the  20  JPME  research 
institutions  and  14  other  DOD-funded  research  institutions  included  in 
the  scope  of  our  review. 

•  Third,  for  all  the  JPME  and  DOD-funded  research  institutions 
conducting  studies  and  analysis  research,  we  collected  a  list  of 
research  projects  they  conducted  for  fiscal  years  2012  and  2013.  To 
complete  the  content  analysis,  one  GAO  analyst  independently 
reviewed  each  of  the  2,217  research  project  titles  and  coded  them 
into  one  or  more  of  the  23  areas  of  concentration.  When  the  coding 
was  completed,  two  GAO  analysts  shared  responsibility  to  review  the 
coding  made  by  the  first  analyst  and  indicated  whether  they  agreed  or 
disagreed  with  the  code.  The  analysts  then  met  to  discuss  their 
coding  determinations  and  to  reach  agreement  where  there  were  any 
discrepancies.  The  results  of  our  analysis  are  not  generalizable 
beyond  the  20  JPME  research  institutions  and  14  other  DOD-funded 
research  institutions  included  in  the  scope  of  our  review. 

•  Fourth,  we  reviewed  documentation  about  the  offices  that  request 
research  from  the  JPME  and  other  DOD-funded  research  institutions, 
along  with  testimonial  evidence  gathered  during  our  interviews  with 
DOD  officials  to  provide  context  for  any  similarities  or  dissimilarities 
we  identified  through  the  mission  statement  and  project  title  analysis. 

To  further  our  understanding  of  DOD’s  processes  for  requesting  research 

and  of  the  similarities  and  differences  among  research  organizations,  we 

conducted  interviews  with  officials  from  the  Joint  Staff;  Office  of  the  Under 

Secretary  of  Defense  (Comptroller);  Office  of  the  Under  Secretary  of 
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Defense  for  Policy;  Office  of  the  Secretary  of  Defense,  Studies  and 
Federally  Funded  Research  and  Development  Center  Management 
Office;  Office  of  the  Assistant  Secretary  of  Defense  for  Research  and 
Engineering;  Army  Study  Program  Management  Office;  Air  Force 
Analyses,  Assessments  and  Lessons  Learned  office;  Office  of  the  Chief 
of  Naval  Operations;  Marine  Corps  Analysis  Directorate;  National 
Defense  University;  Army  War  College;  Army  Command  and  General 
Staff  College;  Air  University;  Naval  War  College;  and  Marine  Corps 
University. 

We  conducted  this  performance  audit  from  February  2013  through  March 
2014  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 
that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives. 
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This  appendix  contains  more-detaiied  information  for  each  of  the  20  joint 
professionai  miiitary  education  (JPME)  research  institutions  inciuded  in 
the  scope  of  our  review.  For  each  research  institution,  we  provide  a  one- 
page  summary  that  inciudes  information  on  the  foiiowing  eiements: 

•  Location:  the  institution’s  associated  JPME  coiiege  or  university  and 
geographicai  iocation. 

•  Background:  the  date  that  the  institution  was  estabiished  and 
information  on  its  estabiishment,  as  weii  as  any  reievant  information 
such  as  changes  in  the  research  institution’s  name. 

•  Mission:  the  institution’s  mission,  either  seif-reported  or  as  derived 
from  reievant  documentation. 

•  Customers:  a  iist  of  entities,  as  reported  by  the  research  institutions 
and  other  documents  that  represent  the  principai  requesters  and 
users  of  the  research  institutions’  research  products. 

•  Nature  of  research  and  pubiications:  a  summary  description  of  the 
types  of  research  and  studies  conducted  by  the  research  institution, 
as  weii  as  the  names  of  pubiications  produced  by  the  institution,  if 
any. 

•  Totai  funding:^  the  institution’s  totai  funding,  depicted  in  thousands  of 
doiiars,  forfiscai  years  2004  to  2013,  as  avaiiabie.  Totai  funding  is 
subdivided  into  two  categories — direct  funding  and  reimbursabie 
funding.  Direct  funding  inciudes  federai  appropriations  made  avaiiabie 
for  JPME  coiieges  or  universities.  Reimbursabie  funding  refers  to 
amounts  earned  or  coiiected  from  outside  offices  for  research 
services  furnished  by  the  institution. 


^We  assessed  the  reliability  of  the  funding  and  staffing  data  collected  by  analyzing 
questionnaire  responses  from  JPME  colleges  and  universities,  which  included  information 
on  their  data-system  management,  data  quality-assurance  processes,  and  potential 
sources  of  errors  and  mitigations  of  those  errors.  Based  on  our  review  of  the  data 
provided  and  our  review  of  the  questionnaire  responses,  we  concluded  that  the  systems 
used  to  provide  the  data,  and  thus  the  data  they  provide,  are  sufficiently  reliable  for  our 
audit  purposes.  Furthermore,  although  we  have  identified  a  number  of  factors  that  could 
affect  data  quality,  we  concluded  that  these  were  the  best  available  data  on  JPME 
research  institutions. 
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•  Total  staffing:  the  institution’s  total  number  of  personnel,  expressed  as 
full-time  equivalents^  for  fiscal  years  2004  through  2013,  as  available. 
Full-time  equivalents  are  calculated  as  the  total  hours  worked  divided 
by  the  number  of  compensable  hours  in  a  full-time  schedule. 


^DOD  reported  staffing-level  data  in  full-time  equivalents,  which  are  calculated  as  the  total 
hours  worked  in  jobs  retained  divided  by  the  number  of  hours  in  a  full-time  schedule,  and 
include  noncontractor  researchers,  contractor  researchers,  and  noncontractor 
administrative  staff  positions. 
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Location:  Air  University, 
Montgomery,  AL 

Background:  Established  in  2008  by 
a  Special  Order  from  Headquarters 
Air  Force,  the  Air  Force  Research 
Institute’s  research  roots  extend  back 
to  the  Airpower  Research  Institute  of 
the  late  1970s  and  even  the  1930s  in 
the  Air  Corps  Tactical  School.  The 
orgarhzational  functions  were 
previously  embedded  in  the  Air  War 
College  and  later  the  College  of 
Aerospace  Doctrine,  Research  and 
Education. 

Mission:  To  conduct  research, 
outreach,  and  engagement  to 
enhance  national  security  and  assure 
the  effectiveness  of  the  United  States 
Air  Force. 

Examples  of  Customers 

•  Secretary  of  the  Air  Force 

•  Air  Education  and  Training 
Command 

•  Air  University  officials 

•  Combatant  commands 

•  Chief  of  Staff  of  the  Air  Force 

•  National  security  community 


Air  Force  Research  Institute 


Nature  of  Research  and  Publications 


The  Air  Force  Research  Institute  augments  Air  University’s  and  the  Air  Force’s 
research  capacity  and  supports  airpower  research  inquiries  from  the  Chief  of  Staff 
of  the  Air  Force,  as  well  as  other  decision  makers  throughout  DOD.  The  institute 
conducts  research  on  topics  related  to  air,  space,  and  cyberspace  opportunities, 
threats,  and  capabilities;  evaluation  of  operational  and  strategic  issues;  conducts 
regional  strategic  assessments;  estimates  long-term  strategic  and  technical 
capabilities;  analyses  of  logistical  constraints  and  basing  issues,  among  other 
issues.  The  institute  also  serves  as  the  focal  point  and  provides  support  for  Air 
Uruversity’s  “Call  for  Topics,”  which  makes  potential  research  topics  of  interest  to 
Air  Force  leaders  available  to  student  researchers.  The  institute  also  operates  the 
Air  University  Press,  and  publishes  the  Department  of  the  Air  Force’s  Air  and 
Space  Power  Journal,  and  the  Strategic  Studies  Quarterly. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  in  thousands 
3,000 


Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation. 


Fiscal  Year  2004-2013  Total  Staffing 
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Source:  GAO  analysis  of  DOD  data. 
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Lbout  the  Center 


Location:  Air  University, 
Montgomery,  AL 

Backgronnd:  The  Center  for 
Strategy  and  Technology  was 
established  at  the  Air  University  in 
1996. 

Mission:  To  engage  in  long-term 
strategic  thinking  about  the 
implications  of  technological  change 
and  its  implications  for  U.S.  national 
security.  The  Center  for  Strategy  and 
Technology  focuses  on  education, 
research,  and  publications  that 
support  the  integration  of  technology 
into  national  strategy  and  policy. 

Examples  of  Customers 

•  Educational  institutions 

•  Senior  military  and  political 
officials 

•  Think  tanks 


Air  Force  Center  for  Strategy  and  Technology 


Nature  of  Research  and  Publications 


The  Center  for  Strategy  and  Technology  promotes  the  integration  of  technology 
and  strategy  in  support  of  Air  Force  and  U.S.  national  security  objectives. 

The  center  conducts  research  on  topics  such  as  future  concepts  and  technologies 
and  critical  areas  of  emerging  technologies  such  as  directed  energy, 
biotechnology,  nanotechnology,  and  cyber  technologies,  among  others.  The 
center  also  manages  a  series  of  long-range  studies,  called  Blue  Horizons  study, 
looking  20  to  30  years  into  the  future  to  provide  a  vision  to  prepare  the  Air  Force 
for  future  challenges.  Research  conducted  under  the  auspices  of  the  center  is 
briefed  to  the  Air  Staff,  published  as  occasional  papers  and  disseminated  to  senior 
military  and  political  officials,  think  tanks,  educational  institutions,  and  other 
interested  parties. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  in  thousands 
1,000 


Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation. 


Fiscal  Year  2004-2013  Total  Staffing 
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Source:  GAO  analysis  of  DOD  data. 
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Location:  Air  University, 
Montgomery,  AL 

Background:  Established  in  1998  at 
the  direction  of  the  Chief  of  Staff  of 
the  Air  Force  by  a  memorandum  of 
agreement  between  the  Air  War 
College  Commandant  and  the  Air 
Staff  Director  for  Nuclear  and 
Counterproliferation. 


Air  Force  Counterproliferation  Center 


Nature  of  Research  and  Publications 


The  center  provides  research  articles,  papers,  and  monographs  addressing  issues 
pertinent  to  U.S.  military-response  options  for  dealing  with  nuclear,  biological, 
and  chemical  threats  and  attacks.  Research  topics  include  military  and  diplomatic 
policy  and  concepts  related  to  weapons  of  mass  destruction;  international 
nonproliferation  diplomacy;  nonproliferation  and  arms  control  treaty  regimes; 
counterterrorist  activities;  and  nuclear  deterrence  of  conflicts.  The  center 
develops  a  strategic-issues  list  that  provides  potential  research  topics  to  student 
and  faculty  researchers.  Additionally,  it  conducts  outreach  on  issues  related  to 
counterproliferation  and  nuclear  operations  through  its  publication  of 
Counterproliferation  Center  Outreach  Journal,  and  the  Trinity  Site  Papers 
series,  and  through  an  annual  conference  on  countering  weapons  of  mass 
destruction. 


Fiscal  Year  2004-2013  Total  Funding 


Mission:  To  develop  Air  Force, 
Department  of  Defense,  and  other 
U.S.  Government  leaders  who 
advance  the  state  of  knowledge, 
policy,  and  practices  within  strategic 
defense  issues  involving  nuclear, 
biological,  and  chemical  weapons. 

Examples  of  Customers 

•  Defense  Threat  Reduction 
Agency 

•  National  Nuclear  Security 
Administration 

•  Headquarters  Department  of 
the  Air  Force 

•  Air  Force  major  commands 

•  U.S.  Strategic  Command 

•  Center  for  the  Study  of 
Weapons  of  Mass 
Destruction 


Dollars  in  thousands 
3,000 


Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation. 


Fiscal  Year  2004-2013  Total  Staffing 


Full-time  equivalents 
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Source:  GAO  analysis  of  DOD  data. 
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L\bout  the  Center 


Location:  Army  Command  and 
General  Staff  College,  Leavenworth, 
KS 

Background:  The  Center  for  Army 
Leadership  was  established  in  2001  in 
response  to  a  study  chartered  by  the 
Chief  of  Staff  of  the  Army  to  identify 
the  characteristics  and  skills  required 
for  Army  leaders  in  light  of  changes 
to  the  operational  environment. 

Mission:  To  conduct  leadership  and 
leader  development  research,  studies, 
analysis,  assessment  and 
evaluation;  to  provide  the  Army 
leadership  and  leader  development 
doctrine  products  and  services;  to 
develop  and  maintain  the  Army 
Leader;  to  manage  the  Army  Leader 
Development  Program. 


Center  for  Army  Leadership 


Nature  of  Research  and  Publications 


The  Center  for  Army  Leadership  conducts  research  and  studies  to  identify  leader 
development  trends  and  requirements,  and  to  develop  and  promote  leadership 
and  leader  development  practices  and  techniques  for  the  Army.  Further,  the 
Center  accomplishes  these  outcomes  by  contributing  to  Army  doctrine  and  policy 
by  informing  leadership  on  best  practices  for  developing  leader  competencies, 
and  by  producing  the  Annual  Survey  of  Army  Leadership,  which  is  a  survey- 
based  study  that  assesses  Army-leader  attitudes  regarding  leader  education, 
including  the  quality  of  leadership,  and  the  contribution  of  leadership  to 
accomplishing  the  Army’s  overall  mission. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  in  thousands 
9,000 


Fiscal  year 

Source:  GAO  analysis  of  [)OD  data. 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation.  Army  Command  and 
General  Staff  College  officials  were  unable  to  provide  reliable  data  prior  to  fiscal  year  2007. 


Fiscal  Year  2004-2013  Total  Staffing 


Examples  of  Customers 

•  Joint  Staff 

•  Combatant  commands 

•  Military  services 

•  Army  commands 

•  Other  Department  of  Defense 
Organizations 


Full-time  equivalents 
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Source:  GAO  analysis  of  DOO  data. 

Note:  Army  Command  and  General  Staff  College  officials  were  unable  to  provide  reliable  data  prior  to 
fiscal  year  2007. 
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•TWDllc 


Lbout  the  Center 


Location:  Army  Command  and 
General  Staff  College,  Leavenworth, 
KS 

Backgronnd:  The  Combat  Studies 
Institute  was  established  in  1979  to 
provide  a  range  of  military  historical 
and  educational  support  to  the 
Combined  Arms  Center,  Training  and 
Doctrine  Command,  and  the  United 
States  Army. 

Mission:  To  provide  military 
historical  and  educational  support  to 
the  Combined  Arms  Center,  Training 
and  Doctrine  Command,  and  the 
United  States  Army. 

Examples  of  Customers 

•  Army  Leadership 

•  Army  Training  and  Doctrine 
Command 


Combat  Studies  Institute 


Nature  of  Research  and  Publications 


The  Combat  Studies  Institute  researches,  writes,  and  publishes,  through  the 
Combat  Studies  Institute  Press,  original  interpretive  works  on  doctrinal  and 
operational  issues  of  relevance  to  the  U.S.  Army  and  policymakers.  The  center 
also  implements  U.S.  Army  Training  and  Doctrine  Command’s  program  of  military 
history  instruction  throughout  the  Army,  conducts  an  oral  history  research 
program  that  targets  Command  and  General  Staff  College  students  and  faculty,  as 
well  as  visitors  of  the  Combined  Arms  Center,  focusing  on  compiling  their  past 
operational  experiences,  and  provides  oversight  responsibilities  for  the  Combined 
Arms  Center  Command  History  program,  the  Staff  Ride  team — which  offers  live 
and  virtual  battlefield  tours — and  the  Frontier  Army  Museum. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  in  thousands 
5,000 


Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation.  Army  Command  and 
General  Staff  College  officials  were  unable  to  provide  reliable  data  prior  to  fiscal  year  2007. 


Fiscal  Year  2004-2013  Total  Staffing 


Full-time  equivaients 
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Source:  GAO  analysis  of  DOD  data. 

Note:  Army  Command  and  General  Staff  College  officials  were  unable  to  provide  reliable  data  prior  to 
fiscal  year  2007. 


Page  57 


GAO-14-216  JPME  Research 


-  Appendix  II:  Joint  Professional  Military  Edncation  Research  Institntions  - 


STRATEGIC 

STUDIES 

INSTITUTE 


L^bout  the  Center 


Location:  Army  War  College,  Carlisle 
Barracks,  PA 

Background:  Established  in  1954  by 
the  Commandant  of  the  Army  War 
College  to  create  an  advanced  study 
group  to  undertake  a  program  of 
long-range  thinking  on  strategy  and 
land  power. 

Mission:  To  conduct  and 
disseminate  independent  strategic 
analysis  that  develops 
recommendations  for  addressing  key 
national  security  issues. 

Examples  of  Customers 

•  Office  of  the  Secretary  of 
Defense 

•  Joint  Staff 

•  Combatant  commands 

•  Military  services 

•  U.S.  Army  Headquarters 

•  Army  commands 


Strategic  Studies  Institute 


Nature  of  Research  and  Publications 


The  Strategic  Studies  Institute  conducts  research  on  topics  such  as  the  future  of 
American  strategy;  geostrategic  analyses;  strategic  landpower;  Army  forward 
presence  in  the  Pacific;  cyber  security;  energy  security;  Army’s  role  in  missile 
defense;  effects  of  war  on  leadership;  and  Army  profession  and  public  trust.  The 
institute  also  compiles  a  Key  Strategic  Issues  List  based  on  input  from  the  U.S. 
Army  War  College  faculty,  the  Army  Staff,  the  Joint  Staff,  the  unified  and  specified 
commands,  and  other  Army  organizations.  This  is  designed  to  guide  the  research 
of  the  Strategic  Studies  Institute,  the  U.S.  Army  War  College,  and  other  Army- 
related  strategic  analysts. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  in  thousands 
4,500 


Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation. 


Fiscal  Year  2004-2013  Total  Staffing 


Full-time  equivalents 
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Source:  GAO  analysis  of  DOD  data. 
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Location:  Marine  Corps  University, 
Quantico,  VA 

Background:  The  Translational 
Research  Group  was  established  in 
2010  by  the  Director  of  the  Center  for 
Advanced  Operational  Culture  and 
Learning  at  the  Marine  Corps 
University  and  by  the  Executive 
Deputy  of  the  Marine  Corps  Training 
and  Education  Command. 

Mission:  To  identify  practical 
applications  for  social  and  behavioral 
scientific  research  that  will  help 
address  pressing  challenges  facing 
the  Marine  Corps. 

Examples  of  Customers 

•  Marine  Corps  leadership 

•  Marine  Corps  combat 
personnel 


Translational  Research  Gronp 


Nature  of  Research  and  Publications 


The  Translational  Research  Group  at  the  Center  for  Advanced  Operational 
Culture  Learning  aims  to  link  the  findings  of  scientists  with  the  needs  of  Marine 
Corps  soldiers  and  leadership  by  helping  the  two  sides  understand  each  other’s 
needs  and  capabilities. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  in  thousands 
2,000 


Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation. 


Fiscal  Year  2004-2013  Total  Staffing 
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Source;  GAO  analysis  of  DOD  data 
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Location:  Marine  Corps  University, 
Quantico,  VA 


History  Division 


Nature  of  Research  and  Publications 


The  History  Division’s  primary  task  is  to  research  and  write  the  Marine  Corps’ 
official  history.  The  division  provides  assistance  through  their  reference  branch 
and  deploying  field  historians  to  record  history  in  the  making  during  operations. 
The  History  Division  and  also  conducts  research  through  an  oral  history  program, 
in  which  it  obtains,  catalogs,  transcribes,  and  preserves  personal  narrative 
experiences  and  observations  of  historic  value  from  active-duty  and  retired 
Marines  for  use  as  reference  source  material.  The  division  prepares  a  wide  variety 
of  official  publications  that  tell  the  Marine  Corps  story.  Publications  include 
articles,  monographs,  occasional  papers,  and  definitive  histories.  It  also  creates 
material  for  and  publishes  Fortitudine,  an  online  bulletin  of  the  Marine  Corps 
history  program. 


Background:  Established  in  1919  by 
Marine  Corps  Headquarters  to 
record,  preserve,  and  distribute  the 
Corps’  history,  the  History  Division 
was  transferred  in  2005  to  Marine 
Corps  University. 

Mission:  To  write,  document  and 
track  the  history  of  the  Marine  Corps 
across  the  entire  spectrum  of  time;  to 
collect  documents  and  accounts  of 
permanent  value  to  the  history  of  the 
Marine  Corps  and  preserve  them  for 
future  use;  and  distribute  the  history 
of  the  Corps  through  publications, 
papers  and  other  programs,  in  order 
to  preserve  history,  aid  combat  and 
noncombat  decision  making,  support 
Professional  Military  Education, 
motivate  Marines,  and  inform  the 
American  public. 

Examples  of  Customers 

•  Joint  Staff 

•  Marine  Corps  personnel 

•  White  House 

•  U.S.  citizens 

•  Members  of  Congress 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  in  thousands 


Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation. 


Fiscal  Year  2004-2013  Total  Staffing 
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Source;  GAO  analysis  of  DOD  data 
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L^bout  the  Center 


Location:  Marine  Corps  University, 
Quantico,  VA 

Background:  Established  in  2007  at 
the  request  of  the  Commanding 
General  of  the  Marine  Corps  Combat 
Development  Command.  The  Marine 
Corps  University  Education 
Command  Order  issued  in  2011 
included  the  Middle  East  Studies 
Center  as  one  of  the  university’s  staff 
offices. 

Mission:  To  serve  as  the  Marine 
Corps’  center  of  expertise  on  the 
Middle  East,  and,  more  broadly, 
Pakistan  and  Afghanistan,  in  order  to 
deepen  the  Marine  Corps’ 
imderstanding  of  this  critical  region 
and  to  link  the  Marine  Corps  to  the 
broader  academic, 
intergovernmental,  and  international 
Middle  East  studies  community. 

Examples  of  Customers 

•  Marine  Corps  command 
elements 

•  Marine  Corps  leadership 

•  Military  services 

•  Federal  agencies 


Middle  East  Studies 


Nature  of  Research  and  Publications 


In  an  effort  to  improve  the  Marine  Corps’  understanding  of  the  complex  security 
environment  of  the  Middle  East,  the  center  began  three  forms  of  publications.  The 
Middle  East  Studies  Occasional  Paper  Series,  with  the  first  issue  published  in 
June  2011,  aims  to  disseminate  original,  peer-reviewed  research  papers  on  a  wide 
variety  of  subjects  pertaining  to  the  Middle  East,  Afghanistan,  and  Pakistan.  The 
MES  Monograph  Series,  with  the  first  issue  published  in  August  2011,  focuses  on 
subjects  of  strategic  relevance  to  the  current  and  future  U.S.  professional  military 
education  community  and  is  meant  to  be  published  quickly  to  address  fast- 
developing  situations.  Finally,  the  Middle  East  Studies  institute  publishes 
Insights,  which  is  produced  bimonthly  as  the  newsletter  of  the  center.  It  features 
short  analytical  pieces  as  well  as  information  on  events  organized  by  the  center 
and  provides  a  forum  for  debate  with  readers. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  in  thousands 


Fiscal  year 

Source:  GAO  analysis  of  DOD  data 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation. 


Fiscal  Year  2004-2013  Total  Staffing 


Full-time  equivalents 
2.0 


1.5 


1.0 


0.5 


0.0  " 

2004  2005  2006  2007  2008  2009  2010  2011  2012  2013 

Fiscal  year 

Source:  GAO  analysis  of  DOO  data 


Page  61 


GAO-14-216  JPME  Research 


-  Appendix  II:  Joint  Professional  Military  Edncation  Research  Institntions  - 


cco 

DIPLOMACY  •  DEFENSE  •  DEVELOPMENI 
Lbout  the  Center 


Location:  National  Defense 
University,  Washington,  DC 

Backgronnd:  The  U.S.  Congress 
authorized  the  creation  of  a  Center 
for  Complex  Operations  in  the 
National  Defense  Authorization  Act 
for  fiscal  year  2009.  The  center  was 
initially  established  in  2008  in  the 
Office  of  the  Under  Secretary  of 
Defense  for  Policy,  and  moved  in 
early  2009  to  the  National  Defense 
University  in  accordance  with  a 
memorandum  of  agreement  between 
the  National  Defense  University  and 
the  Office  of  the  Under  Secretary  of 
Defense  for  Policy,  dated  January  30, 
2009,  subsequently  amended. 

Mission:  To  foster  unity  of  effort 
among  Department  of  Defense  and 
interagency  personnel  in  complex 
operations;  to  collect  and  analyze 
lessons  from  military  and  civilian 
personnel;  and  to  incorporate  those 
lessons  into  policy,  doctrine, 
education,  training,  and  exercises. 

Examples  of  Customers 

•  Office  of  the  Under  Secretary 
of  Defense  for  Policy 

•  Joint  Staff 

•  Combatant  Commands 

•  U.S.  Department  of  State 

•  U.S.  Agency  for  International 
Development 


Center  for  Complex  Operations 


Nature  of  Research  and  Publications 


The  center  collects  and  analyzes  interagency  lessons  from  the  field  on  overseas 
contingency  operations,  including  stabilization,  irregular  warfare,  and  security 
assistance,  and  integrates  them  into  joint  military  doctrine  on  such  topics  as 
counterinsurgency,  stability  operations,  security  cooperation,  and  interagency 
coordination;  as  well  as  into  education,  policy,  training,  and  joint 
military/interagency  exercises.  The  center  also  analyzes  interagency  aspects  of 
overseas  operations  on  behalf  of  Department  of  Defense,  the  Intelligence 
Community,  and  several  federal  agencies.  The  center’s  principal  journal,  PRISM, 
serves  to  inform  members  of  U.S.  federal  agencies,  allies,  and  other  partners  on 
complex  and  integrated  national  security  operations;  reconstruction  and  nation¬ 
building;  relevant  policy  and  strategy;  lessons  learned;  and  developments  in 
training  and  education.  The  center  also  produces  publications  on  issues  of 
importance  to  interagency  stakeholders  and  JPME,  such  as  its  recent  book 
Convergence:  Elicit  Networks  in  the  Age  of  Globalization. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  in  thousands 

3.500 

3,000  P'''®*:* 

2.500 

2,000  Reimbursable  funding 

1.500 
1,000 

500 
0 

2004  2005  2006  2007  2008  2009  2010  2011  2012  2013 

Fiscal  year 

Source:  GAO  arralysis  of  OOD  data. 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation. 


Fiscal  Year  2004-2013  Total  Staffing 


Full-time  equivalents 
20 

15 

10 

5 

0 

2004  2005  2006  2007  2008  2009  2010  2011  2012  2013 

Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 

Note:  National  Defense  University  officials  were  unable  to  provide  reliable  staffing  level  data  for  the 
Center  for  Complex  Operations  prior  to  fiscal  year  2010.  According  to  the  National  Defense 
University  officials,  prior  year  staffing  level  data  for  the  center  were  captured  in  summary-level  information 
provided  to  us  for  the  Institute  for  National  Security  Studies,  which  is  the  parent  organization  for  the 
National  Defense  University’s  research  institutions. 
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Conflict  Records  Research  Center 


Nature  of  Research  and  Publications 


To  fulfill  its  mission,  the  Conflict  Records  Research  Center  was  charged  with 
organizing,  encouraging,  and  facilitating  greater  analytic  and  academic  access  to 
digitized  copies  of  captured  documents;  coordinating  the  translation  of  captured 
documents  of  interest;  conducting  sponsored  research  and  analysis  on  captured 
documents;  developing  and  delivering  training  programs  and  providing  research 
assistance;  informing  researchers  of  the  collections  in  its  custody;  and  publishing 
research  in  books,  reports,  journal  articles,  corrference  papers,  newsletters,  or 
other  media. 


L\bout  the  Center 


Fiscal  Year  2004-2013  Total  Funding 


Location:  National  Defense 
University,  Washington,  DC 

Background:  Established  in  2009  at 
the  direction  of  the  Office  of  the 
Under  Secretary  of  Defense  for 
Policy  as  a  subelement  of  the 
Institute  for  National  Strategic 
Studies.  The  center  was  defunded  for 
fiscal  year  2014,  but  in  September 
2013,  the  Office  of  the  Under 
Secretary  of  Defense  for  Policy 
provided  funding  to  keep  the  center 
open.  Section  1071  of  the  National 
Defense  Authorization  Act  for  Fiscal 
Year  2014  provided  statutory 
authorization  for  the  center. 

Mission:  To  facilitate  research  using 
records  captured  during  combat 
operations  from  coimtries, 
organizations,  and  individuals  now  or 
once  hostile  to  the  United  States. 


Dollars  in  thousands 
2,000 


Fiscal  year 

Source;  GAO  analysis  of  DOD  data. 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation. 


Fiscal  Year  2004-2013  Total  Staffing 


Full-time  equivalents 
6 

S 

4 

3 

2 


Examples  of  Customers 

•  Office  of  the  Under  Secretary 
of  Defense  for  Policy 

•  Joint  Staff 

•  Combatant  commands 

•  Federal  agencies 


2004  2005  2006  2007  2008  2009  2010  2011  2012  2013 

Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 
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CSR 

CENTER  FOR 

STRATEGIC  RESEARCH 

Lbout  the  Center 


Location:  National  Defense 
University,  Washington,  DC 

Backgronnd:  Established  in  1984  by 
the  Secretary  of  Defense  as  the 
Research  Directorate  of  National 
Defense  University’s  Institute  for 
National  Security  Studies,  the  center 
was  originally  charged  to  provide 
independent  advice  to  Secretary  of 
Defense,  the  Chairman  of  the  Joint 
Chiefs  of  Staff,  and  the  combatant 
commands  for  the  formulation  of 
national  security  policy  and  strategy. 
The  center  was  renamed  in  2010 
dming  the  university’s  research 
reorganization. 

Mission:  To  provide  educational 
support  to  joint  professional  military 
education  and  advice  to  the  Secretary 
of  Defense,  Chairman  of  the  Joint 
Chiefs  of  Staff,  and  the  combatant 
commands  through  studies,  reports, 
briefings,  and  memorandums; 
conducts  directed  research  and 
analysis  in  the  areas  of  strategic 
studies  and  regional  studies;  and 
engages  in  independent  and  leading- 
edge  research  and  analysis  in  related 
areas. 

Examples  of  Customers 

•  Office  of  the  Secretary  of 
Defense 

•  Chairman  of  the  Joint  Chiefs 
of  Staff 


•  Combatant  commands 

•  Federal  agencies 

•  Military  services 


Center  for  Strategic  Research 


Nature  of  Research  and  Publications 


The  Center  for  Strategic  Research  performs  research  and  educational  activities  in 
support  of  joint  professional  military  education  and  explores  strategic  and 
regional  topics  to  offer  advice  and  strategic  support  to  the  Office  of  the  Under 
Secretary  of  Defense  for  Policy,  the  Joint  Staff,  and  other  senior  DOD  officials. 

The  center  also  conducts  outreach  to  share  its  research  with  policymakers 
through  studies,  reports,  briefings,  and  memorandums.  Strategic  studies 
encompass  national  security  and  military  strategy,  to  include  defense  policy  and 
organization,  deterrence,  arms  control  and  counter  proliferation,  peace  operations 
and  small-scale  contingencies,  transnational  security  problems,  command  and 
control,  and  future  warfare.  Regional  studies  encompass  national  security 
strategy,  defense  policy,  defense  cooperation,  and  military  strategy  issues  as  they 
relate  to  significant  countries  or  geographic  areas  of  the  world  such  as  Asia  and 
the  Middle  East.  The  center’s  publication  product  line  includes  books.  Occasional 
Papers,  Strategic  Perspectives,  Strategic  Forum  (policy  papers),  conference 
papers,  and  journal  articles. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  In  thousands 
3,500 


Fiscal  year 

Source;  GAO  analysis  of  DOD  data 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation. 


Fiscal  Year  2004-2013  Total  Staffing 


Full-time  equivalents 


10 


5 


2004  2005  2006  2007  2008  2009  2010  2011  2012  2013 

Fiscal  year 

Source:  GAO  analysis  of  DOD  data 
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INSS 


ISST  ro^  SATtOHAL 

strategic  studies 


L^bout  the  Center 


Location:  National  Defense 
University,  Washington,  DC 

Background:  Established  in  2000  as 
a  part  of  National  Defense 
University’s  Institute  for  National 
Strategic  Studies,  pursuant  to  the 
National  Defense  Authorization  Act 
for  Fiscal  Year  2000. 

Mission:  To  serve  as  a  national  focal 
point  and  resource  center  for 
multidisciplinary  research  and 
analytic  exchanges  on  the  national 
goals  and  strategic  posture  of  the 
People’s  Republic  of  China  and  the 
ability  of  that  nation  to  develop,  field, 
and  deploy  an  effective  military 
instrument  in  support  of  its  national 
strategic  objectives. 

Examples  of  Customers 

•  Office  of  the  Secretary  of 
Defense 

•  Joint  Staff 

•  Military  services 

•  U.S.  Pacific  Command 

•  Federal  agencies 


Center  for  the  Study  of  Chinese  Military  Affairs 


Nature  of  Research  and  Publications 


The  center’s  research  focuses  on  documenting  China’s  expanding  international 
interests,  understanding  China’s  development  and  employment  of  new  economic, 
military,  and  diplomatic  capabilities,  and  analyzing  Chinese  debates  about  how 
these  capabilities  should  be  employed  to  advance  national  goals.  It  also  explores 
the  implications  of  these  developments  for  U.S. -China  relations  and  for  the  U.S. 
role  in  Asia.  The  center  also  conducts  outreach  to  share  its  research  with 
policymakers  and  informs  the  public  debate  through  books,  articles, 
memorandums,  briefings  and  conferences.  For  example,  the  center  cosponsors  an 
armual  corrference  on  the  People’s  Liberation  Army  with  the  Council  for  Advanced 
Policy  Studies  (a  Taiwanese  think  tank),  and  RAND,  a  nonprofit  institution  that 
conducts  research  and  analysis.  The  center’s  publication  product  line  includes 
books.  Occasional  Papers,  Strategic  Perspectives,  Strategic  Forum  (policy 
papers),  conference  papers,  and  journal  articles. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  in  thousands 
800 


Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation. 


Fiscal  Year  2004-2013  Total  Staffing 


Full-time  equivalents 
3 


2 


1 


2004  2005  2006  2007  2008  2009  2010  2011  2012  2013 

Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 
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K  Center  for  the  Study  of  § 


VWD 


^  Weapons  of  Mass  Destruction  ^ 


hout  the  Center 


Location:  National  Defense 
University,  Washington,  DC 

Backgronnd:  Established  in  1994  as 
the  Center  for  Counterproliferation 
Research,  pursuant  to  memoranda  of 
imderstanding  among  the  Assistant 
Secretary  of  Defense  for  Nuclear 
Security  and  Counterproliferation, 
the  Director  of  Strategic  Plans  and 
Policy,  the  Joint  Staff,  and  the 
President,  National  Defense 
University.  The  center  was  renamed 
the  Center  for  the  Study  of  Weapons 
of  Mass  Destruction  in  2004. 

Mission:  To  prepare  the  joint 
warfighter  and  select  others  to 
address  the  challenges  posed  by 
weapons  of  mass  destruction  through 
education  and  professional 
development,  scholarship,  and 
outreach  and  collaboration  activities 
across  the  full  spectrum  of  issues 
related  to  weapons  of  mass 
destruction  and  to  become  one  of  the 
preeminent  institutions  in  the  United 
States  for  weapons  of  mass 
destruction  expertise. 


Center  for  the  Study  of  Weapons  of  Mass 
Destruction 


Nature  of  Research  and  Publications 


The  Center  for  the  Study  of  Weapons  of  Mass  Destruction  performs  research  on 
the  full  spectrum  of  issues  related  to  weapons  of  mass  destruction,  engages  in 
educational  activities,  and  collaborates  with  partners  across  the  government.  The 
center  conducts  directed  and  self-initiated  research  of  the  following  types:  (1) 
operational  and  policy  support;  (2)  traditional  academic  research;  and  (3) 
research  undertaken  to  support  the  joint  professional  military  education  program 
at  the  National  Defense  University.  Research  topics  related  to  studies  in  weapons 
of  mass  destruction  include  deterrence;  counterproliferation  operations;  and 
policy  and  doctrinal  development  regarding  weapons  of  mass  destruction.  The 
center  also  conducts  outreach  to  share  its  research  through  papers,  and  plarming 
and  participating  in  various  venues,  including  conferences  and  dialogues  with 
participants  from  U.S.  and  foreign  partner  entities.  For  example,  the  center 
organizes  the  annual  meeting  of  the  Weapons  of  Mass  Destruction  Education 
Consortium. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  in  thousands 


Fiscal  year 

Source:  GAO  analysis  of  000  data 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation. 


Examples  of  Customers 

•  Office  of  the  Secretary  of 
Defense 

•  Joint  Staff 

•  Combatant  commands 

•  Intelligence  commimity 

•  Federal  agencies 


Fiscal  Year  2004-2013  Total  Staffing 


Full-time  equivalents 
20 


10 


5 


2004  2005  2006  2007  2008  2009  2010  2011  2012  2013 

Fiscal  year 

Source:  GAO  analysis  of  DOD  data 
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ibout  the  Center 


Location:  National  Defense 
University,  Washington,  DC 

Background:  The  Center  for 
Technology  and  National  Security 
Policy  was  created  by  the  President 
of  the  National  Defense  University  in 
2001  to  address  technology  and 
policy  gaps  which  affect  national 
security  policy  and  decision  making. 

Mission:  To  research  national 
security  and  defense  policies  in  the 
context  of  transformational  changes 
in  emerging  technology,  advanced 
concepts,  organizational  structures, 
and  international  trends  to  develop 
and  implement  actionable 
recommendations,  to  publish  and 
distribute  findings,  and  to  introduce 
research  results  in  the  joint 
professional  military  education  class 
room. 

Examples  of  Customers 

•  Office  of  the  Secretary  of 
Defense 

•  Under  Secretary  of  Defense  for 
Policy 

•  Under  Secretary  of  Defense  for 
Acquisitions,  Technology,  and 
Logistics 

•  Joint  Staff 

•  Military  services 


Center  for  Technology  and  National  Secnrity  Policy 


Nature  of  Research  and  Publications 


The  Center  for  Technology  and  National  Security  Policy  conducts  research  on  a 
reimbursable  basis  by  means  of  memorandums  of  understanding  with  sponsoring 
organizations  on  science  and  technology  (chemical/biological  defense,  human 
hardiness  research,  counter-improvised  explosive  devices,  policing,  and 
counterinsurgency);  civilian-military  integration  (transformative  irmovation  for 
development  and  emergency  support,  social  media  in  strategic  communication); 
emerging  challenges  (anticipatory  governance  concerning  cyber  security;  climate 
change;  vulnerability  to  severe  space  weather);  and  advanced  education 
initiatives.  The  center’s  publication  product  line  includes  books,  Defense 
Technology  Papers,  Defense  Horizons  (policy  papers),  conference  papers,  and 
journal  articles. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  In  thousands 


Fiscal  year 

Souioa  GAO  analysis  ol  CXX>  data 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation. 


Fiscal  Year  2004-2013  Total  Staffing 
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TRANSATLANTIC 
SECURITY  STUDIES 


Center  for  Transatlantic  Secnrity  Stndies 


Nature  of  Research  and  Publications 


L\bout  the  Center 


Location:  National  Defense 
University,  Washington,  DC 

Background:  The  Center  for 
Transatlantic  Security  Studies  was 
formed  in  2010  to  support  the  Under 
Secretary  of  Defense  for  Policy  in 
North  Atlantic  Treaty 
Organization/European-related 
policy  development.  The  center 
incorporated  the  NATO  Orientation 
Program  that  had  provided  training  to 
NATO  assigned  officers  since  at  least 
1990  as  was  mandated  by  the 
Chairman  of  the  Joint  Chiefs  of  Staff. 
As  of  October  2012,  both  the  center 
and  the  NATO  Orientation  Program 
were  dissolved. 

Mission:  To  be  the  focal  point  for 
national  and  international 
collaboration  on  issues  related  to 
transatlantic  security,  defense  policy, 
and  military  strategy  through 
research,  education,  and  outreach. 
Develops  and  conducts  education 
and  orientation  programs  for  U.S.  and 
allied  military  officers,  govermnent 
civilians,  and  international  partners 
on  issues  relating  to  NATO  and 
transatlantic  security  community. 

Examples  of  Customers 

•  Under  Secretary  of  Defense  for 
Policy 

•  Joint  Staff 

•  U.S.  European  Command 

•  North  Atlantic  Treaty 
Organization 

•  Allied  Command  Transformation 


The  Center  for  Transatlantic  Studies  provided  senior  Department  of  Defense  and 
other  U.S.  government  leaders  with  the  North  Atlantic  Treaty  Organization 
(NATO)  and  Transatlantic  policy  advice,  research,  and  outreach,  notably  in  the 
run-up  to  the  2012  NATO  Summit  and  beyond.  Specifically,  the  center  conducted 
research  on  capabilities  studies,  evaluating  transatlantic  bargain  and  dialogue, 
and,  NATO-Russia  relations,  and  NATO’s  countering  hybrid  threats.  It  also 
published  Transatlantic  Currents,  CTSS  Flash  notes,  and  Transatlantic 
Perspectives. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  In  thousands 
1.500 


Fiscal  year 

Sourcii'  GAO  analysis  ol  DOD  data 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation. 


Fiscal  Year  2004-2013  Total  Staffing 


Full-time  equivalents 
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Source:  GAO  analysis  o(  DOD  data. 
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Location:  Naval  War  College, 
Newport,  R1 


China  Maritime  Studies  Institute 


Nature  of  Research  and  Publications 


The  China  Maritime  Studies  Institute^  has  four  primary  areas  of  activity:  (1)  broad, 
multidisciplinary  research  on  China’s  maritime  activity  as  it  relates  to  its  strategic 
orientation;  (2)  armual  conferences  and  speaker  series;  (3)  publications,  ranging 
from  short  assessments  and  think  pieces  to  monographs  and  books;  and  (4) 
support  for  U.S.  Navy  and  joint  commands.  The  center  conducts  research  in  areas 
related  to  China’s  maritime  development,  including  energy,  global  commerce,  law 
of  the  sea,  maritime  technologies,  merchant  marine,  naval  development,  naval 
diplomacy,  and  shipbuilding. 


Fiscal  Year  2004-2013  Total  Funding 


Background:  The  China  Maritime 
Studies  Institute  was  established  as  a 
subcomponent  of  the  Strategic 
Research  Department  on  October  1, 
2006,  in  accordance  with  a  Program 
Objective  Memorandum  by  the  Chief 
of  Naval  Operations. 

Mission:  To  increase  knowledge  and 
imderstanding  of  the  maritime 
dimensions  of  China’s  rise;  evaluate 
the  open-source  maritime  literature 
in  China;  provide  analyses  for  the 
Navy,  and  maintain  collegial 
relationships  with  Chinese  national 
security  scholars 

Examples  of  Customers 

•  U.S.  Navy  leadership 

•  Military  leaders 

•  Intelligence  community 

•  Federal  agencies 


Dollars  In  thousands 
1,500 


Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation.  Naval  War  College  officials 
were  unable  to  provide  reliable  data  prior  to  fiscal  year  2007. 


Fiscal  Year  2004-2013  Total  Staffing 


Full-time  equivalents 
7 
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0  - 
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Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 

Note:  Naval  War  College  officials  were  unable  to  provide  reliable  data  prior  to  fiscal  year  2007. 

Vitnougn  the  China  Maritime  studies  institute  was  established  as  a  subcomponent  of  the  Strategic 
Research  Department,  it  receives  dedicated  funding  on  an  annual  basis  for  its  research  activities, 
including  to  fund  its  own  researchers.  As  a  result,  we  have  categorized  the  China  Maritime  Studies 
Institute  as  a  separate  joint  professional  military  education  research  institution  for  the  purposes  of  this 
report.  However,  because  the  center  is  a  subcomponent  of  the  Strategic  Research  Department, 
Naval  War  College  officials  stated  that  it  shares  budgetary,  personnel,  and  administrative  functions 
with  the  Strategic  Research  Department. 
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About  the  Center 


Location:  Naval  War  College, 
Newport,  R1 

Background:  The  International  Law 
Department  was  founded  in  1984  and 
opened  its  doors  in  1986. 

Mission:  To  conduct  and 
disseminate  advanced  international 
law  research  and  analysis. 

Examples  of  Customers 

•  U.S.  Navy  leadership 

•  Federal  agencies 

•  Military  leadership 

•  Naval  War  College 


International  Law  Department 


Nature  of  Research  and  Publications 


The  International  Law  Department  serves  as  the  Naval  War  College’s  focal  point 
for  the  study  of  international  and  maritime  law  and  oceans  policy  as  they  affect 
U.S.  military  policy,  strategy  and  operations.  As  part  of  its  research  efforts,  the 
department  compiles,  edits,  and  publishes  the  International  Law  Studies  Series 
that  provides  a  forum  for  prominent  legal  scholars  to  publish  articles  that 
contribute  to  the  broader  understanding  of  international  law.  Recently,  in 
response  to  discussion  with  the  Joint  Staff,  the  department  initiated  an 
Information  Paper  Series.  These  are  short  papers  that  break  down  legal  issues  for 
further  consideration  by  senior  military  leaders.  Furthermore,  in  addition  to  the 
Information  Paper  Series  and  International  Law  Studies,  the  individual  staffers 
of  the  department  engage  in  independent  research  and  writing.  In  addition  to  legal 
research  and  scholarly  writing.  Naval  War  College  officials  state  that  staff 
members  actively  support  the  Naval  War  College’s  core  intermediate  and  senior- 
level  Navy  PME  courses  as  professors,  lecturers,  and  moderators,  while  hosting 
several  operational-law  electives  throughout  the  academic  year. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  In  thousands 
1,000 


Fiscal  year 

Source:  GAO  analysis  of  DOD  data 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation.  Naval  War  College  officials 
were  unable  to  provide  reliable  data  prior  to  fiscal  year  2007. 


Fiscal  Year  2004-2013  Total  Staffing 
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Source:  GAO  analysis  of  DOD  data 

Note:  Naval  War  College  officials  were  unable  to  provide  reliable  data  prior  to  fiscal  year  2007. 
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About  the  Center 


Location:  Naval  War  College, 
Newport,  R1 

Background:  Formally  established  in 
1987  by  the  Dean  of  the  Center  for 
Naval  Warfare  Studies,  and  in 
consultation  with  the  Chief  of  Naval 
Operations.  However,  the  Strategic 
Research  Department  dates  back  to 
the  origins  of  the  Center  for  Naval 
Warfare  Studies  in  1981. 

Mission:  To  produce  innovative 
strategic  research  and  analysis  for 
the  U.S.  Navy,  Department  of 
Defense,  and  the  broader  national 
security  community. 

Examples  of  Customers 

•  Secretary  of  the  Navy 

•  Chief  of  Naval  Operations 

•  U.S.  Marine  Corps  leadership 

•  Combatant  commands 

•  Federal  agencies 


Strategic  Research  Department 


Nature  of  Research  and  Publications 


The  Strategic  Research  Department’s  research  projects,  including  those  of  the 
China  Maritime  Studies  Institutes,  fall  into  three  broad  categories:  (1)  projects 
assigned  by  the  Navy  or  another  U.S.  national  security  organization;  (2)  sustained 
projects  that  do  not  depend  on  year-to-year  tasking  but  rather  constitute 
multiyear,  multideliverable,  multiclient  investments  serving  long-term  U.S. 
national  security  and  Navy  interests  (these  projects  focus  on  strategic  regions 
such  as  Eurasia,  Africa,  the  greater  Middle  East  and  the  Asia-Pacific  region,  as 
well  as  on  functional  topics  such  as  maritime  strategy,  cyber  conflict,  and  sea- 
based  ballistic  missile  defense);  (3)  self-sponsored  projects  conducted  in 
consultation  with  the  leadership  of  the  Center  for  Naval  Warfare  Studies  (these 
projects  have  no  specific  tasking  or  set  of  clients,  rather,  they  address  emerging 
issues  that  officials  beheve  will  gamer  substantial  national  or  naval  attention  in 
the  foreseeable  future). 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  In  thousands 
2,000 


Fiscal  year 

Source:  GAO  analysis  of  DOD  data 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation.  Naval  War  College  officials 
were  unable  to  provide  reliable  data  prior  to  fiscal  year  2007. 


Fiscal  Year  2004-2013  Total  Staffing 


Full-time  equivalents 
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Source:  GAO  analysis  of  DOD  data. 

Note:  Naval  War  College  officials  were  unable  to  provide  reliable  data  prior  to  fiscal  year  2007. 
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Lbout  the  Center 


Location:  Naval  War  College, 
Newport,  R1 

Backgronnd:  The  Strategic  Studies 
Group  was  established  by  the  Chief 
of  Naval  Operations  in  1981.  The 
Strategic  Studies  Group  is  only 
tasked  by  and  reports  directly  to  the 
Chief  of  Naval  Operations. 

Mission:  To  generate  revolutionary 
naval  warfare  concepts  for  the  future 

Examples  of  Customers 

•  Chief  of  Naval  Operations 


Strategic  Studies  Group 


Nature  of  Research  and  Publications 


Each  year,  the  Chief  of  Naval  Operations  selects  a  broad  governing  theme  for  the 
Strategic  Studies  Group’s  research.  The  2013  topic  is  Undersea  Dominance  out  to 
2030.  The  Strategic  Studies  Group  is  responsible  for  keeping  the  Chief  of  Naval 
Operations  irrformed  of  progress  throughout  the  year  and  produces  a  summary 
briefing  and  written  report  of  actionable  concepts  with  recommendations  that  can 
be  executed  by  the  Chief  of  Naval  Operations  in  the  near  term.  The  products, 
while  encompassing  long-term  views,  are  designed  to  help  inform  the  Chief  of 
Naval  Operations  on  near-  and  mid-term  program  decisions. 


Fiscal  Year  2004-2013  Total  Funding 


Dollars  in  thousands 
4,000 


Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 

Note:  The  reported  funding  is  in  nominal  dollars,  unadjusted  for  inflation.  Naval  War  College  officials 
were  unable  to  provide  reliable  data  prior  to  fiscal  year  2007. 


Fiscal  Year  2004-2013  Total  Staffing 


Full-time  equivalents 


°  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013 

Fiscal  year 

Source:  GAO  analysis  of  DOD  data. 

Note:  Naval  War  College  officials  were  unable  to  provide  reliable  data  prior  to  fiscal  year  2007. 
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This  appendix  contains  the  results  of  our  research  project  title  analysis. 
We  reviewed  2,217  research  project  titles  from  20  joint  professional 
military  education  research  institutions  and  14  other  DOD-funded 
research  organizations.  These  titles  are  associated  with  projects 
conducted  in  2012  and  2013.  Based  on  a  project’s  title,  we  coded  it  into 
one  or  more  areas  of  concentration.  The  table  below  presents  the  number 
of  research  project  titles  coded  into  these  23  areas  of  concentration.  The 
results  do  not  add  to  2,217  because  some  research  project  titles  could  be 
coded  into  more  than  one  area  of  concentration  and  two  areas  of 
concentration  are  not  included  in  the  final  results.^  For  example,  a 
research  project  title  on  North  Korea’s  nuclear  future  would  be  coded  into 
both  the  “Asia”  and  “nuclear  and  weapons  of  mass  destruction”  topic 
areas. 


^The  results  do  not  include  the  following  two  areas  of  concentration:  (1)  Other;  and  (2) 
Unable  to  Code.  “Other”  and  “Unable  to  Code”  were  not  included  because  these  are  not 
areas  of  concentration  intended  to  show  similarity.  Rather,  “Other”  is  intended  for  project 
titles  that  do  not  fit  into  the  other  areas  of  concentration,  and  “Unable  to  Code”  is  used  for 
methodological  purposes  to  categorize  incomplete  information. 
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Figure  7:  Similarities  in  Research  Topic  Areas  for  34  Research  Organizations,  According  to  2012  and  2013  Research  Project 
Titles 
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Source  GAO  analysts  of  DOD  inform3tk>n 

'’Army  Command  and  General  Staff  College. 
Note:  See  abbreviations  list  on  page  iii. 
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Appendix  IV:  Comments  from  the 
Department  of  Defense 


THE  JOINT  STAFF 

WASHINGTON,  DC 

6  March  2014 


Ms.  Johana  Ayers 
Acting  Director 

Defense  Capabilities  and  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington,  DC  20548 

Dear  Ms.  Ayers: 

The  Department  of  Defense  has  reviewed  draft  report  GAO- 14-2 16,  “Joint  Professional 
Military  Education:  Opportunities  Exist  for  Greater  Oversight  and  Cooidination  of  Associated 
Research  Institutions,”  ^ed  29  January  2014  (GAO  351808).  We  concur  with  the  two 
recommendations  to  more  reliably  assess  the  perfoimance  of  JPME  research  institutions. 

Substantive  wodc  has  already  begun  to  clarify  organizational  goals  and  establish  metrics 
of  success  at  each  of  the  research  institutions.  For  example,  the  Joint  Staffs  Joint  Force 
Development  Directorate  is  collaborating  with  the  National  Defense  University  (NDU)  to  refine 
the  university’s  research  enterprise.  Such  GAO-recommended  efforts  are  reflected  in  the  NDU 
Strategic  Plan  for  Research  2014-2019  and  in  a  revision  of  the  Chairman’s  policy  document  for 
NDU.  They  should  be  fully  implemented  when  the  next  academic  year  convenes  in  fall  2014. 

We  will  concentrate  oversight  of  a  Department-wide  effort  to  improve  coordination  by 
establi^ng  a  DoD  Studies  and  Analysis  Executive  Committee  with  appropriately  defined 
regional  and  topical  “communities  of  interest”  by  the  end  of  FY  2014.  This  executive  committee 
will  be  a  combined  effort  through  the  Under  Secretary  of  Defense  for  Policy  and  the  Assistant 
Secretary  of  Defense  for  Research  and  Engiiu^ring,  with  appropriate  representation  from  the 
PME  and  JPME  enterprise. 

On  behalf  of  the  Secretary  of  Defense  and  Chairman  of  the  Joint  Chiefs  of  Staff,  1 
appreciate  your  efforts  to  highlight  better  use  of  resources  in  development  of  future  DoD  leaders. 

Sincerely, 


cc: 

USD(AT&L) 
USD(P) 
USD(P&R) 
Services  Chiefs 
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Acknowledgments 


GAO  Contact 


Johana  Ayers,  (202)  512-5741  or  ayersj@gao.gov 


Staff 

Acknowledgments 


In  addition  to  the  contact  named  above,  Matthew  Ullengren,  Assistant 
Director;  Erin  Behrmann;  Richard  Burkard;  Gabrielle  A.  Carrington; 
Alberto  Left;  Marcus  Lloyd  Oliver;  Michael  Silver;  and  Cheryl  Weissman 
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GAO’S  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  website  (http://www.gao.gov).  Each  weekday 
afternoon,  GAO  posts  on  its  website  newly  released  reports,  testimony, 
and  correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted 
products,  go  to  http://www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  website, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates.  Listen  to  our  Podcasts. 
Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 
7125,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngc1@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 

Please  Print  on  Recycled  Paper. 

